State-based bilingual model modification for nonnative speech recognition

The speech recognition accuracy has been observed to decrease for nonnative speakers, especially those who are just beginning to learn foreign language or who have heavy accents. This paper presents a novel bilingual model modification approach to improve nonnative speech recognition via considering these great variations of accented pronunciations. Each state of the baseline nonnative acoustic models is modified with several candidate states from the auxiliary acoustic models, which are trained by speakerspsila mother language. State mapping criterion and n-best candidates are investigated based on a grammar-constrained speech recognition system. Using the state-based bilingual model modification approach, compared to the nonnative acoustic models which have already been well trained by adaptation technique MAP, a relative reduction of 11.7% in phrase error rate (RPhrER) was further achieved.