Class document frequency as a learned feature for text categorization

Document classification uses different types of word weightings as features for representation of documents. In our findings we find the class document frequency, dfc, of a word is the most important feature in document classification. Machine learning algorithms trained with dfc of words show similar performance in terms of correct classification of test documents when compared to more complicated features. The importance of dfc is further verified when simple algorithms developed solely on the basis of df c shows performance that compares closely with that of more complex machine learning algorithms. We also found improved performance when the dfc of links of documents in a class is used along with the dfc of the words of the document. We compared the algorithms for showing the importance of dfc on the Reuters-21578 text categorization test classification and the Cora data set.