The Performance Appraising of Language Models and the Entropy Estimation of Chinese

We give a quantified reasoning and description of the perplexity for evaluating language models using the concept of entropy in information theory: The smaller the entropy of the language estimated by the language model is, the more precise the language model is; an interpolated model based on two (n-1)-gram models is better than the (n-1)-gram component models, but not a n-gram model. We also explore the methods to estimating the entropy of Chinese using language models.