Predictability score-based feature selection

The greater than ever amount of data used in predictive data mining calls for new and flexible approaches, based on soft computing methods, with the purpose of recognizing the valuable attributes in datasets. The selection of relevant features is an important part of the preprocessing step in pattern recognition, statistics, knowledge discovery and data mining. Discovering attributes with little or without predictive information allows overfitting avoidance and comprehensibility improvement of the resulting model. In the literature, there are several measures to estimate the quality of the attributes. This paper proposes a predictability-based feature selection technique, in concert with a fuzzy inference system, to point out the potential variables in predictive modeling. The same as Relief family methods, this approach takes into account the context of other attributes, given the target value, to assess the predictability of attributes according to how well their values distinguish between instances that are near each other. Using Bayesian decision theory and fuzzy rules, the system provides, for each feature, its relevancy, as qualitative information and its predictability score as quantitative measure. Encouraging first results presented in two case studies conclude this paper, bringing experimental evidence to support our proposal. However, theoretical and experimental investigations must continue until better feature selection strategies will emerge. Our work is just another step toward this goal.