A Novel Hierarchical Searching Cluster Strategy for Lazy Learning

Lazy learning is a memory-based learning technique that defers processing of the dataset until it receives request for prediction. The idea is to store all observations from a process in a database and then to estimate a local model by selecting portions of data that belong to a small neighborhood around the current operating point. The bottleneck of lazy learning is management of large datasets and searching for the relevant neighbors. This paper aims to this question and the k-means cluster algorithm is cast into lazy learning framework. Using this strategy, the nearest neighbors searching process can be turned into hierarchical searching process with two levels and a lot of searching time for lazy learning can be saved. Another contribution of this paper is that a new similar criterion between two examples is advanced. Using this criterion, the k-vector nearest neighbors (k-VNN) technique is used to find the region surrounding the query point and the updating database strategy can be obtained without need any further computation. Furthermore, this updating strategy can save a lot of memory space and decrease the neighbors searching process. The advantages of those methodologies are demonstrated on nonlinear function prediction.