Feature selection via minimizing nearest neighbor classification error

Feature selection is viewed as an important preprocessing step for pattern recognition, machine learning and data mining. It is used to find an optimal subset to reduce computational cost, increase the classification accuracy and improve result comprehensibility. In this paper, a weighted distance learning approach is introduced to minimize Leaving-One-Out classification error using a gradient descent algorithm. The quality of features is evaluated with the learned weight and the features with great weights are considered to be useful for classification. Experimental analysis shows that the proposed approach has better performance than several state-of-the-art methods.