Performance Prediction Model in Heterogeneous MapReduce Environments

Map Reduce has emerged as a popular computing model for parallel processing of cloud computing. Map Reduce performance analysis and modeling is needed to guide performance optimization and job scheduling. However, we observed that it is difficult to build a performance model due to various aspects of workload behavior and heterogeneity among cluster nodes in heterogeneous Map Reduce Environments. To address the above issues, in this paper, we propose a novel performance prediction model for Map Reduce in heterogeneous environments. This model consists of two components: (1) performance prediction model based on machine learning and (2) optimal parameters selection based on immune algorithm. Experiment results show that our model can accurately forecast the performance of Map Reduce jobs that run in heterogeneous Map Reduce systems.