This paper presents a generative approach to speech enhancement based on a recurrent variational autoencoder (RVAE). The deep generative speech model is trained using clean speech signals only, and it is combined with a nonnegative matrix factorization noise model for speech enhancement. We propose a variational expectation-maximization algorithm where the encoder of the RVAE is fine-tuned at test time, to approximate the distribution of the latent variables given the noisy speech observations. Compared with previous approaches based on feed-forward fully-connected architectures, the proposed recurrent deep generative speech model induces a posterior temporal dynamic over the latent variables, which is shown to improve the speech enhancement results.
Audio examples
You can listen to randomly picked audio examples for 4 types of noise and 3 signal-to-noise (SNR) ratios (computed using the ITU-R BS.1770-4 protocol). Just click on the links in the table below.
Xavier Alameda-Pineda acknowledges the French National Research Agency (ANR) for funding the ML3RI project.
This work has been partially supported by MIAI @ Grenoble Alpes, (ANR-19-P3IA-0003).
