SVM ensembles for selecting the relevant feature subsets

In this paper we present a novel feature selection algorithm for SVMs which works by estimating the stability of a feature´s contribution to some evaluation criterion. This algorithm is extremely fast as only a small number of SVM classifiers need to be trained for feature selection. Robust results are shown with toy as well as real-life datasets. Furthermore, we combine this method with a backward elimination procedure. The combined algorithm performs stably and shows optimal performance compared with other feature selection methods. Another merit of the combined algorithm is that it can estimate the optimal number of features with the best prediction power. This method is applicable to both linear and nonlinear problems.