New feature selection methods based on context similarity for text categorization

High dimensionality of the feature space is one of the most important concerns in text categorization problems, and feature selection is widely used for reducing the dimensionality of features to speed up the computation without damaging the performance. However, a lot traditional feature selection methods treat each feature separately, and they are context independent. In order to address the problem, this paper first presents the study of four well known frequency based feature selection methods, including Gini Index (GI), Document Frequency (DF), Class Discriminating Measure (CDM) and Accuracy Balanced (Acc2). Then we focus on calculating the importance of features through measuring the similarity of their contexts among the documents but the document frequency containing these features to incorporate context information. Hence we propose four new context similarity based feature selection methods, GI_cs, DF_cs, CDM_cs and Acc2_cs. They are evaluated on different data sets and compared against the four corresponding frequency based methods. Through experimental analysis, the results reveal that the context similarity based methods outperform the corresponding frequency based methods in terms of the micro and macro F1 measures both on binary and multi-classification problems. Benefit from the multi-words information surrounding features, the context similarity based feature selection methods are effective for article categorization.