Title :
LiPS: A cost-efficient data and task co-scheduler for MapReduce
Author :
Ehsan, Mehdi ; Yao Chen ; Hui Kang ; Sion, Radu ; Wong, Johnson
Author_Institution :
Stony Brook Univ., Stony Brook, NY, USA
Abstract :
We introduce LiPS, a new cost-efficient data and task co-scheduler for MapReduce in a cloud environment. By using linear programming to simultaneously co-schedule data and tasks, LiPS helps to achieve minimized dollar cost globally. We evaluated LiPS both analytically and on Amazon EC2 in order to measure actual dollar charges. The results were significant; LiPS saved 62-81% of the dollar costs when compared with the Hadoop default scheduler and the delay scheduler, while also allowing users to fine-tune the cost-performance tradeoff.
Keywords :
cloud computing; parallel programming; scheduling; Amazon EC2; Hadoop default scheduler; LiPS scheduler; MapReduce; cloud environment; cost-performance tradeoff; data-task coscheduler; delay scheduler; dollar charges; dollar costs; linear programming; Cloud computing; Data transfer; Delays; Lips; Processor scheduling; Schedules; Throughput;
Conference_Titel :
High Performance Computing (HiPC), 2013 20th International Conference on
Conference_Location :
Bangalore
DOI :
10.1109/HiPC.2013.6799103
