Title :
Load balancing in MapReduce environments for data intensive applications
Author :
Yang Liu ; Maozhen Li ; Alham, N.K. ; Hammoud, S. ; Ponraj, M.
Author_Institution :
Sch. of Eng. & Design, Brunel Univ., Uxbridge, UK
Abstract :
The distributed computations are widely used in the modern world for processing large scale jobs. Hadoop framework which is based on Google MapReduce model becomes popular due to its great processing power and ease to use. However, due to lack of load management, especially in a heterogeneous computing environment, the performance of Hadoop framework may be deteriorated. Therefore this paper presents a load balancing algorithm which aims to balance the load among heterogeneous nodes. And also, the Hadoop simulator HSim is involved to evaluate the performance of the load balancing algorithm. The results indicate that the performance of the cluster has been significantly enhanced.
Keywords :
distributed processing; resource allocation; Google MapReduce model; HSim; Hadoop framework; Hadoop simulator; data intensive applications; heterogeneous computing environment; load balancing; load management; Algorithm design and analysis; Clustering algorithms; Genetic algorithms; Hard disks; Load management; Mathematical model; Program processors; Distributed computing; HSim; Hadoop; Load balancing; MapReduce;
Conference_Titel :
Fuzzy Systems and Knowledge Discovery (FSKD), 2011 Eighth International Conference on
Conference_Location :
Shanghai
Print_ISBN :
978-1-61284-180-9
DOI :
10.1109/FSKD.2011.6020071