Improved speech recognition using discriminative integration of multiple local classifiers in lattice rescoring

Model combination is a popular technique to integrate several knowledge sources into automatic speech recognition for better system accuracy. In this paper, we report our recent work on the integration of the hidden Markov model based acoustic model, the multi-layer perceptron based phoneme classifier and Gaussian mixture model based tone classifier in lattice rescoring. Moreover, we use discriminative model weight training to tune the impact of the heterogeneous models according to different phonetic contexts for better model interpolation. Experimental results on continuous mandarin speech recognition show a 8.2% improvement can be obtained using the combination of the three models. We have also evaluated four context dependent weighting schemes using discriminative trained scaling factors. It is also shown by introducing left final dependent contexts, a 4.1% further recognition error reduction can be further obtained.