The improvement and implementation of distributed item-based collaborative filtering algorithm on Hadoop

In the background of big data era, after analyzing the traditional collaborative filtering algorithm, this paper proposes improved item-based collaborative filtering algorithm using “hotweight” as the weight, which is aimed at improving the accuracy of the algorithm and overcoming the defects such as rarefaction and cold-starting. We distribute the algorithm with MapReduce framework, and apply it to the distributed cluster platform Hadoop. This paper adopts real data set to run the algorithm and the experiment´s result expresses that the improved algorithm can run efficiently on the large amounts of data with the better accuracy, and at the same time, can overcome the cold-starting drawback successfully.