Improved Feature Selection Algorithm Based on Concentration and Dispersion

This paper analyzes the concentration and dispersion of the integrated feature selection algorithm (TFFS),and finds their shortcomings: it is difficult for concentration to measure the weigh of the low frequent terms; dispersion ignores the impact of term whose mutual information is negative. Propose a modified feature selection algorithm (TFFSL), which makes certain improvements on concentration and dispersion, and takes the length of terms as a measure of weight factors. The SVM classification experimental results show that: compared with TFFS algorithm, TFFSL algorithm has higher accuracy and more capacity of eliminating irrelevant terms.