An Improved Document Classification Approach with Maximum Entropy and Entropy Feature Selection

Document classification is an important task in the field of document management. Bayesian model needs the feature independent assumption; artificial neural network suffers from the overfitting problem; support vector machine (SVM) does not do well in real-value feature. This paper proposes to combine entropy and machine learning techniques for document classification. Firstly, the cross entropy and average mutual information are presented to effectively extract the features. Secondly, the support vector machine and maximum entropy model is adopted respectively to perform the classification task in the feature space. Furthermore, an improved feature description instead the binary feature with the real-value is presented in this text, since the prior knowledge of each word is helpful to document classification. Finally, we compare our method with the traditional methods, and the experiment showed our method increased 2.78 % F-measures than basic ME model, and 0.95% than naive Bayes model which was smoothed by Good-Turing algorithm.