Title :
A Parameter Dynamic-Tuning Scheduling Algorithm Based on History in Heterogeneous Environments
Author :
Zhao, Xu ; Dong, Xiaoshe ; Cao, Haijun ; Fan, Yuanquan ; Zhu, Huo
Author_Institution :
Dept. of Comput. Sci. & Technol., Xi´´an Jiaotong Univ., Xi´´an, China
Abstract :
In MapReduce model, the job execution time was prolonged by the straggler tasks in heterogeneity environments. The LATE scheduler has introduced the longest remaining time strategy, but it also has some drawbacks such as inaccurate estimated time and the wasting of system resources. In order to solve these problems, we propose two main algorithms : The parameter dynamic-tuning algorithm based history estimates progress of a task accurately since it dynamically tunes the weight of each phase of a map task and a reduce task according to the historical values of the weights, The evaluation-scheduling algorithm reduce the wasting of system resources by evaluating the free slot before launching a straggler task on this node. The two main algorithms are implemented in hadoop 0.20.1. The environment results are satisfaction to our expects and significantly reduce the wasting of system resources.
Keywords :
cloud computing; scheduling; Hadoop 0.20.1; LATE scheduler; MapReduce model; evaluation-scheduling algorithm; heterogeneous environment history; job execution time; map task; parameter dynamic-tuning scheduling algorithm; reduce task; straggler tasks; task history estimation; Algorithm design and analysis; Heuristic algorithms; History; Prediction algorithms; Scheduling; Scheduling algorithms; Tuning; Hadoop; MapReduce; dynamic-tuning;
Conference_Titel :
ChinaGrid Annual Conference (ChinaGrid), 2012 Seventh
Conference_Location :
Beijing
Print_ISBN :
978-1-4673-2623-0
Electronic_ISBN :
978-0-7695-4816-6
DOI :
10.1109/ChinaGrid.2012.24