Robust speech recognition based on structured modeling, irrelevant variability normalization and unsupervised online adaptation

We present a new approach to robust speech recognition based on structured modeling, irrelevant variability normalization (IVN) and unsupervised online adaptation (OLA). In offline training stage, a set of generic HMMs for basic speech units relevant to phonetic classification is trained along with several sets of feature transforms with different degrees of freedom by using a maximum likelihood (ML) IVN-based training strategy. In recognition stage, after a first-pass recognition, the most appropriate set of feature transforms is identified and adapted under ML criterion by using the unknown utterance itself, which is recognized again to achieve better performance by using the adapted feature transforms and the pre-trained generic HMMs. The effectiveness of the proposed approach is confirmed by evaluation experiments on Finnish Aurora3 database.