A study on different backward feature selection criteria over high-dimensional databases

Feature subset selection has become an expensive process due to the relatively recent appearance of high-dimensional databases. Thus, not only the need has arisen for reducing the dimensionality of these datasets, but also for doing it in an efficient way. We propose a new backward search, where attributes are removed given several smart criteria found in the literature and, besides, it is guided using a heuristic which reduces the cost and needed number of evaluations commonly expected from a backward search. Besides, we do not only propose the design of a new forward-backward algorithm but we also provide an experimental study of different criteria to decide the removal of attributes. The result is a very competitive algorithm which does not exceed the in-practice linear complexity while obtaining selected subsets of features with lower cardinality than other state-of-the-art algorithms.