An adaptive nearest neighbor classifier for noisy environments

The performance of the nearest neighbor (NN) algorithm is known to be highly sensitive to the distance measure used to find the NN of a query pattern. Another problem with the NN algorithm is that its performance degrades rapidly as the number of noisy (i.e., mislabeled) training samples increases. To tackle these problems, in this paper, a novel algorithm is proposed that uses the information from the training data to adapt the distance measure for the application in hand. For this purpose, similar to some other works reported in the literature, a weight is assigned to each training instance. The weight assigned to a training instance is used to calculate the distance of a query pattern with that instance. The major contribution of this paper is in the way that these parameters (i.e., the weights of training instances) are specified. The proposed method specifies the weight of training example based on the point that the entropy measure is minimized. We use some benchmark datasets from UCI repository to show that the scheme is quite successful in improving the performance of the basic NN algorithm. When dealing with noisy data, we show that the proposed scheme achieves significantly better performance in comparison with some other methods proposed in the literature.