Automatic Speech Recognition of Code Switching Speech Using 1-Best Rescoring

In this paper, we propose a novel approach to automatic recognition of code-switching speech. The proposed method consists of two phases: automatic speech recognition, and rescoring. The framework uses parallel automatic speech recognizers for speech recognition. The lattices produced are subsequently joined and rescored to estimate the most probable word sequence. Experiment shows that the proposed approach reduction of more than 5% WER, when tested on English/Malay code switching speech. In addition, the framework has shown to be very robust. Besides, we also propose an acoustic model adaptation approach known as hybrid approach of interpolation and merging to cross adapt acoustic models of different languages to recognize code switching speech. The adapted acoustic models show reduction in WER, when they are used for code switching speech recognition.