Validation of $k$ -Nearest Neighbor Classifiers

This paper presents a method to compute probably approximately correct error bounds for k-nearest neighbor classifiers. The method withholds some training data as a validation set to bound the error rate of the holdout classifier that is based on the remaining training data. Then, the method uses the validation set to bound the difference in error rates between the holdout classifier and the classifier based on all training data. The result is a bound on the out-of-sample error rate for the classifier based on all training data.