Title :
P-WLPA algorithm research on parallel framework Spark
Author :
Shuang Wang ; Tao Liu ; Peng Wu ; Wei Min Kong
Author_Institution :
Sch. of Comput. Sci. & Technol., Wuhan Univ. of Technol., Wuhan, China
Abstract :
Weighted Label Propagation Algorithm with Probability Threshold (P-WLPA) with typical serial execution prototype is proposed to be applied in data classification. Meanwhile on distributed computing system Spark, parallel P-WLPA algorithm for labeling big data is conducted. The algorithm sets prior conditions when configuring undirected graph and optimizes parameter learning in P-WLPA process. Through experiments, we analyze the relationship between Iterations for convergence, sample stability threshold and error rate. Finally Serial and parallel P-WLPA performance comparison demonstrates the feasibility and efficiency of the parallel P-WLPA algorithm implementation on Spark.
Keywords :
Big Data; graph theory; parallel algorithms; pattern classification; P-WLPA process; Spark parallel framework; big data labeling; convergence; data classification; distributed computing system; error rate; parallel P-WLPA algorithm; parameter learning; sample stability threshold; serial P-WLPA performance; serial execution prototype; undirected graph; weighted label propagation algorithm with probability threshold; Algorithm design and analysis; Big data; Classification algorithms; Clustering algorithms; Labeling; Prediction algorithms; Sparks; P-WLPA algorithm; Parallel computing; Spark; big data;
Conference_Titel :
Information Technology and Artificial Intelligence Conference (ITAIC), 2014 IEEE 7th Joint International
Print_ISBN :
978-1-4799-4420-0
DOI :
10.1109/ITAIC.2014.7065087
