An Effective Feature Selection Method via Mutual Information Estimation

This paper proposes a new feature selection method using a mutual information-based criterion that measures the importance of a feature in a backward selection framework. It considers the dependency among many features and uses either one of two well-known probability density function estimation methods when computing the criterion. The proposed approach is compared with existing mutual information-based methods and another sophisticated filter method on many artificial and real-world problems. The numerical results show that the proposed method can effectively identify the important features in data sets having dependency among many features and is superior, in almost all cases, to the benchmark methods.