A Framework for Cost-Effective Scheduling of MapReduce Applications

Real-time, cost-effective execution of "Big Data" applications on MapReduce clusters has been an important goal for many scientists in recent years. The MapReduce paradigm has been widely adopted by major computing companies as a powerful approach for large-scale data analytics. However, running MapReduce workloads in cluster environments has been particularly challenging due to the trade-offs that exist between the need for performance and the corresponding budget cost. Furthermore, the large number of resource configuration parameters exacerbates the problem, as users must manually tune the parameters without knowing their impact on the performance and budget costs. In this paper, we describe our approach to cost-effective scheduling of MapReduce applications. We present an overview of our framework that enables appropriate configuration of parameters to detect cost-efficient resource allocations. Our early experimental results illustrate the working and benefit of our approach.