A comparison of mutual and fuzzy-mutual information-based feature selection strategies

It is very important to select a small set of relevant features from a high dimensional data set and useful to design either an effective classification or prediction model. This procedure involves a series of estimations of the relationship between each pair of variables and between each variable and class labels. Mutual information is widely used to estimate these relationships. However, alternative strategies may be useful to estimate the mutual information with continuous or hybrid data. In this study, we attempt to evaluate the difference between the selection strategies involved with mutual information and fuzzy mutual information. The results indicate that using fuzzy mutual information is more helpful to obtain more stable feature sets and more accurate estimations of the relationship between two variables.