The effects of whispered speech on state-of-the-art voice based biometrics systems

In this paper, automatic speaker verification using whispered speech is explored. In the past, whispered speech has been shown to convey relevant speaker identity and gender information, nevertheless it is not clear how to efficiently use this information in speech-based biometric systems. This study compares the performance of three different speaker verification systems trained and tested under different scenarios and with two different feature representations. First, we show the benefits of using AM-FM based features as well as their effectiveness for i-vectors extraction. Second, for the classical mel-frequency cepstral coefficient (MFCC) features we show that gains of up to 40% could be achieved with the fusion of traditional Gaussian mixture model (GMM) based systems and more recent i-vector based ones, relative to using either system alone for normal speech. Additionally, for MFCC, fusion schemes were shown to be more robust to addition of whispered speech data during training or enrollment. Overall, AM-FM based features were shown to be more robust to varying training/testing conditions and to improve speaker verification performance for both normal and whispered speech by using the GMM based system alone.