A Novel Language Model Based on Cognition Attention Attenuation in Web Retrieval

Language model is widely used in many retrieval systems. Its document representation is based on the bag of words assumption. Hence, each term in document is treated as an equal object and only the term frequency is considered as the evidence of the importance of term. In this paper, we study the problem of cognition attention attenuation in processing documents and present a cognition attention attenuation based language model. This model estimates the document model by attenuation process of term in document. Compared with the classical language model, the advantage of this model is considering about the document structure which is often used in text summarization. From the experiments results, our novel cognition attention attenuation based language model outperformed the classical language model with Dirichlet smoothing in blog page and Web page.