Estimating Uncertainty to Improve Exemplar-Based Feature Enhancement for Noise Robust Speech Recognition

We present a method of improving automatic speech recognition performance under noisy conditions by using a source separation approach to extract the underlying clean speech signal. The feature enhancement processing is complemented with heuristic estimates of the uncertainty of the source separation, that are used to further assist the recognition. The uncertainty heuristics are converted to estimates of variance for the extracted clean speech using a Gaussian Mixture Model based mapping, and applied in the decoding stage under the observation uncertainty framework. We propose six heuristics, and evaluate them using both artificial and real-world noisy data, and with acoustic models trained on clean speech, a multi-condition noisy data set, and the multi-condition set processed with the source separation front-end. Taking the uncertainty of the enhanced features into account is shown to improve recognition performance when the acoustic models are trained on unenhanced data, while training on enhanced noisy data yields the lowest error rates.