An evaluation of existing and new feature selection metrics in text categorization

Text categorization is widely used for organizing and manipulating the documents in the electronic medium. Since the data in text categorization field are high-dimensional, feature selection is crucial to make the task more efficient and precise. In this paper, we make an extensive evaluation of the feature selection metrics used in text categorization by using local and global policies. For the experiments, we use three datasets which vary in size, complexity and skewness. We use SVM as the classifier and tf-idf weighting for term weighting. We observed that almost in all metrics, local policy outperforms when the number of keywords is low and global policy outperforms as the number of keywords increases. In addition to the evaluation of the existing feature selection metrics, we propose new metrics, which have shown high success rates especially in datasets with a low number of keywords. Moreover, we propose a keyword selection policy called Adaptive Keyword Selection (AKS). It is based on selecting different number of keywords for different classes and it improved the performance significantly in skew datasets.