An improved feature selection approach based on ReliefF and Mutual Information

A fundamental problem in machine learning is to discriminate a representative set of features on which to construct a classification model for a particular task. This paper presents a feature selection algorithm RF-MI for multiple classes based on ReliefF algorithm and Mutual Information (MI) measure. RF-MI algorithm gets a feature subset by excluding irrelevant and redundant features from original features based on ReliefF algorithm and MI measure respectively, adjusting the feature weight threshold δ and the correlation threshold θ respectively by the classification performance of a specific classifier when using newly generated feature subsets on training data sets, and repeating above procedures until the best classification performance is achieved. Experiments conducted on UCI data sets showed that the presented RF-MI algorithm is better than both ReliefF and GR algorithms in minishing the feature set on the premise that better classification accuracy is maintained. That means classification models based on the feature subset derived from the algorithm can have lower time and space complexity.