An information-theoretic feature selection method based on estimation of Markov blanket

Feature selection is an essential process in computational intelligence and statistical learning. It is often used to reduce the requirement of data measurement and storage and defy the curse of dimensionality in order to improve prediction performance. Although there exist many related works, it remains a challenging problem. In this paper, we first examine a set of desirable characteristics for a good feature selection method and find that most of the existing feature selection methods have fulfilled only part (not all) of these characteristics. We then propose a new feature selection method based on estimation of Markov blanket (FS-EMB) which has all the desirable characteristics. Experimental results based on benchmark data sets show that when combined with different classifiers, FS-EMB performs similar to or better than other state-of-the-art feature selection methods. More over, the performance is stable with a smaller standard deviation with respect to the average performance improvement.