Research and design of an efficient collaborative filtering predication algorithm

Currently collaborative filtering has been widespread used to solve the problem of information overload. However there still remain two major limitations, data sparsity and scalability. We explore a new collaborative filtering algorithm to solve the problem of data scalability and improve the predication accuracy. It uses a binary tree to store partitioned items. In the process of tree formation, a K-means clustering is used to partition data and create the neighbor of similar items, and then predication based on a smaller item database is performed. Since the preliminary clustering greatly reduces the search space, the search for similar neighbor items will be faster than for the entire database. In addition, the cluster that contains similar items is cohesive, thus it can produce a higher overall accuracy. The experimental results argue that our algorithm obviously outperforms current CF algorithms and it is feasible and efficient.