Energy-efficient Resource Management for QoS-guaranteed Computing Clusters

A cluster computing system in data centers not only improves service availability and performance but also increase power consumption. It is a challenge to increase the performance of a cluster computing system and reduce its power consumption simultaneously. MapReduce has recently evolved in data-intensive parallel computing. It is a programming model for processing large data sets. The implementation of MapReduce typically runs on a large scale of cluster computing systems consisting of thousands of commodity machines simply called MapReduce clusters that results in high power consumption, which is a major concern by service providers such as Amazon and Yahoo. In this research, we consider a collection of cluster computing resources owned by a service provider to host an enterprise application for business customers. We investigate the problem of resource allocation for power management in MapReduce clusters. Specifically, we propose resource allocation approaches to minimizing the mean end-to-end delay of customer jobs or services under the constraints of the energy consumption and the availability of MapReduce clusters and to minimizing the energy consumption of MapReduce clusters under the availability of MapReduce clusters and the mean end-to-end delay of customer jobs or services that play an essential role in the delivery of quality of services (QoS) for customer services.. Numerical experiments demonstrate that the proposed approaches are applicable and efficient to solve these resource allocation problems for power management in MapReduce clusters.