Improving language modeling by using distance and co-occurrence information of word-pairs and its application to LVCSR

This paper reports our study in exploiting the distance and co-occurrence information of word-pairs to improve the n-gram language model. We used these two types of information for modeling the distant context, up to history length of ten. Also we show that the proposed model provides complementary information about the n-gram´s context that is unable to be captured by the n-gram model due to data scarcity. Evaluated on the WSJ and SWB-1 corpora, the proposed model reduced the trigram perplexity up to 11.2% and 6.5% respectively. In an N-best re-ranking task of the Aurora-4 database, our model aided a hexagram model to perform ~9% relatively better in terms of WER.