Title :
Scientific Workflow Makespan Reduction through Cloud Augmented Desktop Grids
Author :
Reynolds, Christopher J. ; Winter, Stephen ; Terstyanszky, Gabor Z. ; Kiss, Tamas ; Greenwell, Pamela ; Acs, Sandor ; Kacsuk, Peter
Author_Institution :
Centre for Parallel Comput., Univ. of Westminster, London, UK
fDate :
Nov. 29 2011-Dec. 1 2011
Abstract :
Scientific workflows are common in biomedical research, particularly for molecular docking simulations such as those used in drug discovery. Such workflows typically involve data distribution between computationally demanding stages which are usually mapped onto large scale compute resources. Volunteer or Desktop Grid (DG) computing can provide such infrastructure but has limitations resulting from the heterogeneous nature of the compute nodes. These constraints mean that reducing the make span of a given workflow stage submitted to a DG becomes problematic. Late jobs can significantly affect the make span, often completing long after the bulk of the computation has finished. In this paper we present a system capable of significantly reducing the make span of a scientific workflow. Our system comprises a DG which is dynamically augmented with an infrastructure as a service (IaaS) Cloud. Using this solution, the Cloud resources are used to process replicated late jobs. Our system comprises a core component termed the scheduler, which implements an algorithm to perform late job detection, Cloud resource management (instantiation and reuse), and job monitoring. We offer a formal definition of this algorithm, whilst we also provide an evaluation of our prototype using a production scientific workflow.
Keywords :
cloud computing; grid computing; resource allocation; scheduling; IaaS cloud; biomedical research; cloud augmented desktop grid; cloud resource management; data distribution; drug discovery; infrastructure as a service cloud; job monitoring; job scheduler; large scale compute resources; late job detection; molecular docking simulation; production scientific workflow; scientific workflow makespan reduction; volunteer grid computing; Bridges; Cloud computing; Computer architecture; Monitoring; Portals; Servers; cloud computing; cloudburst; grid computing; workflow;
Conference_Titel :
Cloud Computing Technology and Science (CloudCom), 2011 IEEE Third International Conference on
Conference_Location :
Athens
Print_ISBN :
978-1-4673-0090-2
DOI :
10.1109/CloudCom.2011.13