LDA-based LM adaptation using latent semantic marginals and minimum discriminant information

We introduce an unsupervised language model (LM) adaptation approach using latent Dirichlet allocation (LDA) and latent semantic marginals (LSM). LSM is a unigram probability distribution over words and is estimated using the LDA model. A hard-clustering method is used to form topics. Each document is assigned to a topic based on the maximum number of words chosen from the topic for that document in LDA analysis. An LDA-adapted model is created using the weighted combination of topic models. The LDA-adapted model is modified by using the LSM as dynamic marginals, and a new adapted model is formed by using the minimum discriminant information (MDI) approach, which minimizes the distance between the new adapted model and the LDA-adapted model. We apply LM adaptation approaches for original and automatic (recognition results after first-pass decoding) transcriptions test data and have seen significant perplexity and word error rate (WER) reductions over a traditional approach.