Learning from neighborhood for classification with local distribution characteristics

The k-nearest neighbor method generates predictions for a particular instance from its neighborhood. It is a simple but effective supervised method for classification. However, the traditional k-nearest neighbor algorithm using the majority voting rule for the class label usually loses a part of useful information in the neighborhood. This paper tries to learn from the neighborhood for more useful information for classification and proposes an improved version of k-nearest neighbor method by heuristically organizing the local distribution characteristics. Different from the traditional methods, the proposed method considers the neighborhood of a query sample from the perspective of local distribution and learns from the neighborhood for local distribution characteristics for classification. We analyze the impact of local distribution characteristics on classification and heuristically develop a formulation to estimate the membership degree, which indicates the level of membership of a query sample to each class; then the query sample is classified to the class which has the highest membership degree with respect to the query sample. Experiments have been conducted on several real data sets; the results support the conclusion that the proposed method is superior to the traditional voting k-nearest neighbor method and comparable with or better than several state-of-the-art methods in terms of classification performance and robustness.