Title :
Ultrasound-coupled semi-supervised nonnegative matrix factorisation for speech enhancement
Author :
Barker, Trevor ; Virtanen, Tuomas ; Delhomme, Olivier
Author_Institution :
Dept. of Signal Process., Tampere Univ. of Technol., Tampere, Finland
Abstract :
We present an extension to an existing speech enhancement technique, whereby the incorporation of easily obtained Doppler-based ultrasound data, obtained from frequency shifts caused by a talker´s mouth movements, is shown to improve speech enhancement results. Noisy speech mixtures were enhanced using semi-supervised nonnegative matrix factorisation (NMF). Ultrasound data recorded alongside the speech is transformed into the spectral domain and used additionally to audio in the mixture to be separated. Speech components are learned from a training set, whilst noise components are estimated from the mixture signal. We show that the ultrasound data can improve source-to-distortion ratios for the enhanced speech, relative to both the non-ultrasound NMF case and an established Wiener filter-based speech enhancement method.
Keywords :
Wiener filters; matrix algebra; speech enhancement; NMF; Wiener filter; frequency shifts; mixture signal; noise components; noisy speech mixtures; source-to-distortion ratios; speech components; speech enhancement technique; ultrasound coupled semisupervised nonnegative matrix factorisation; ultrasound data; Acoustics; Dictionaries; Noise; Speech; Speech enhancement; Ultrasonic imaging; Acoustic Doppler Sensor; Nonnegative Matrix Factorisation; Source Separation; Ultrasound;
Conference_Titel :
Acoustics, Speech and Signal Processing (ICASSP), 2014 IEEE International Conference on
Conference_Location :
Florence
DOI :
10.1109/ICASSP.2014.6853975