Least Squares Support Feature Machine

In many cases, data are represented as high dimensional feature vectors. It makes the feature selection necessary to reduce the computational burden, improve the generalization ability and the interpretability. In this paper, we present a novel feature selection method which is named least squares support feature machine (LS-SFM). In comparison with SVM and LS-SVM, this method has two outstanding properties. Firstly, the convex combinations of basic kernels are used as the kernel and each basic kernel makes use of a single feature. It makes the feature selection problem which can not be solved in the context of SVM transformed into an ordinary multiple parameters learning problem. Secondly, those parameters are learned by a two stage iterative algorithm. A 1-norm based regularized cost function is used to enforce sparsity of feature parameters. The ´support features´ refer to the respective features with nonzero feature parameters. Some UCI datasets are used to demonstrate the effectiveness and efficiency of this approach