Feature Selection Using Recursive Feature Elimination for Handwritten Digit Recognition

In this paper, a new feature selection method with applications to handwritten digit recognition is proposed. This method is based on recursive feature elimination (RFE) in least squares support vector machines (LS-SVM). Digit recognition is achieved by one-against-all LS-SVMs. The RFE method is adapted to multi-class classification in two ways. One is to prune features for each binary LS-SVM classifier independently, and the other is to prune features for all the binary classifiers jointly. The multi-class RFE is also compared with the wrapper feature selection method which uses genetic algorithms. The experimental results indicate that the joint pruning algorithm yields the best performance and selects more features relevant to intrinsic characteristics of digits.