FS_SFS: a novel feature selection method for support vector machines

This paper presents a novel feature selection method which is named filtered and supported sequential forward search (FS_SFS) in the context of support vector machines (SVM). In comparison with conventional wrapper methods employing the sequential forward search (SFS) strategy, it has two important properties that reduce the computation time of SVM training during the feature selection process. First, instead of utilizing all the training samples to obtain the classifier, FS_SFS, by taking advantage of the existence of support vectors in SVM, dynamically maintains an active data set for each SVM to be trained on. In this way, the computational demand of a single SVM training decreases. Secondly, a new criterion, in which the discriminant ability of individual features and the correlation between them are both taken into consideration, is proposed to effectively filter out non-essential features before every SFS iteration begins. As a result, the total amount of training is significantly reduced. The proposed approach is tested on both synthetic and real data to demonstrate its effectiveness and efficiency.