Title :
QoS-aware Resource Provisioning for Big Data Processing in Cloud Computing Environment
Author :
Hassan, Mohammad ; Song, Bo ; Hossain, M. Shamim ; Alamri, Atif
Author_Institution :
Coll. of Comput. & Inf. Sci., King Saud Univ., Riyadh, Saudi Arabia
Abstract :
Nowadays, the quantity of collected data from many different sources is increasing dramatically. As the traditional on-hand computing resources are not sufficient enough to handle Big data, deploying the processing services into clouds is becoming an inevitable trend. For QoS (quality of service)-aware Big data processing, a specially designed cloud resource allocation approach is required. Presently, it is challenging to incorporate the comprehensive QoS demand of Big data with cloud while minimizing the total cost. In order to solve this problem, a general problem formulation is established in this paper. By giving certain assumptions, we prove that the reduction of resource waste has a direct relation with cost minimization. Based on that, we develop efficient heuristic algorithms with tuning parameters to find cost minimized dynamic resource allocation solutions for the above-mentioned problem. In paper, we study and test the workload of Big data by running a group of typical Big data jobs, i.e., video surveillance services, on Amazon Cloud EC2. Then we create a large simulation scenario and compare our proposed method with other approaches.
Keywords :
Big Data; cloud computing; cost reduction; minimisation; quality of service; resource allocation; video surveillance; Amazon Cloud EC2; QoS-aware resource provisioning; big data processing; cloud computing environment; cost minimized dynamic resource allocation solutions; heuristic algorithms; resource waste reduction; tuning parameters; video surveillance services; Bandwidth; Dynamic scheduling; Measurement; Quality of service; Resource management; Servers; Big data; cloud computing; resource allocation;
Conference_Titel :
Computational Science and Computational Intelligence (CSCI), 2014 International Conference on
Conference_Location :
Las Vegas, NV
DOI :
10.1109/CSCI.2014.103