Title :
A parallel algorithm of association rules based on cloud computing
Author :
Wang Yong ; Zhang Zhe ; Wang Fang
Author_Institution :
Dept. of Comput. Sci. & Eng., Guilin Univ. of Electron. Technol., Guilin, China
Abstract :
In view of the traditional parallel FP-growth algorithm (PFP)that suffers from two major limitations, namely, multiple database scans requirement (i.e., high I/O cost) and high inter-processor communications cost, therefore we design and implement a parallel association rules mining method based on cloud computing. The algorithm adopts the separation strategy to simply visit a local database only once, thus, the inter-processor communication I/O overhead is reduced. What´s more, the MapReduce model is used to solve the problem of huge amounts of data mining, as well as the calculated execution taking place in the local data storage node, which can avoid large amounts of data on the network transmission and reduce the communication overhead. By using ordinary PC structures, Hadoop cluster experimental results verify that the proposed algorithm based on cloud computing offers higher efficiency and has a good speedup.
Keywords :
cloud computing; data mining; database management systems; parallel algorithms; Hadoop cluster; MapReduce model; PC structures; PFP; cloud computing; data mining; database scans requirement; interprocessor communication IO overhead; interprocessor communications cost; local data storage node; local database; network transmission; parallel FP-growth algorithm; parallel algorithm; parallel association rules mining method; Algorithm design and analysis; Association rules; Cloud computing; Computational modeling; Itemsets; Association Rules; Cloud Computing; FP-growth; MapReduce; Parallel Computing;
Conference_Titel :
Communications and Networking in China (CHINACOM), 2013 8th International ICST Conference on
Conference_Location :
Guilin
DOI :
10.1109/ChinaCom.2013.6694632