Title :
Implementation of K-Means based on improved storm model
Author :
Jingling Zhao ; Zhaohua Sun ; Qing Liao
Author_Institution :
Beijing Univ. of Posts & Telecommun., Beijing, China
Abstract :
In recent years, big data processing has been a trend. Hadoop and some other cloud computing technologies make batch processing possible to process big data. Storm is like a real-time Hadoop. Storm model is easy, but it´s difficult to deal with complex Topology due to the increasing inter-dependences between components. This paper proposes a novel method which simplifies the Storm Topology programming model by combining Spring with Storm. It provide a modular approach and a unified configuration model to build topologies and easy to use API for using Storm. Meanwhile, this paper propose a system to process data and realize the K-Means clustering algorithm in Storm. The practice results is shown and analyzed to prove the effectiveness of the system and model, at the same time it proves that Storm can improve the clustering algorithm processing speed.
Keywords :
Big Data; application program interfaces; batch processing (computers); cloud computing; pattern clustering; API; Hadoop; K-Means clustering algorithm; Spring; Storm Topology programming model; batch processing; big data processing; cloud computing technologies; Algorithm design and analysis; Clustering algorithms; Fasteners; Global Positioning System; Springs; Storms; Topology; GPS; K-Means; Kafka; Spring; Storm; Stream processing;
Conference_Titel :
Communication Technology (ICCT), 2013 15th IEEE International Conference on
Conference_Location :
Guilin
DOI :
10.1109/ICCT.2013.6820470