Document similarity measure for topic detection in BBS

Document similarity calculation methods are closely related to specific applications. Document-similarity based topic detection in BBS needs to solve two problems: first, to highlight words which are rich in topic information; second, to overcome the adverse impact of huge variance in text length among BBS texts. This paper proposed a novel approach to address the issues: First, the features are divided into five categories: persons (includes organizations), locations, nouns, verbs, others; second, features are selected in each category respectively. Experiments show that the approach yield significant improvement over the traditional way.