Design Dynamic Data Allocation Scheduler to Improve MapReduce Performance in Heterogeneous Clouds

This paper conducts a thorough research on one of the critical technologies in cloud computing, MapReduce programming model. Some of past research results showed that their methods can be executed through allocating identical tasks to each cloud node for enhancing MapReduce performance. However, such allocations are not applicable for the environment of heterogeneous cloud. Due to the different computing power and system resources between the nodes, such uniform distribution of tasks will lower the performance between nodes, and hence this paper makes improvement on the original speculative execution method of Hadoop and LATE Scheduler by proposing a new scheduling scheme known as Dynamic Data Allocation Scheduler (DDAS). DDAS adopts more accurate methods to determine the response time and backup task that affect the system, which is expected to enhance the success ratio of backup tasks and thereby to effectively increase the system ability to respond. Three different simulation experiments are performed and the using of DDAS scheme proves that that DDAS can reduce 30%, 18% and 21% of execution time relative to Hadoop. Also, the DDAS shows a more accurate speculative execution and reasonable allocation of backup tasks. Hence, DDAS can effectively enhance the performance of MapReduce processing in heterogeneous Cloud environment.