GreenPipe: A Hadoop Based Workflow System on Energy-efficient Clouds

Author

Mao, Yaokuan ; Wu, Wenjun ; Zhang, Hui ; Luo, Liang

Author_Institution

State Key Software Dev. Environ. Lab., Beihang Univ., Beijing, China

fYear

2012

fDate

21-25 May 2012

Firstpage

2211

Lastpage

2219

Abstract

Cloud computing is increasingly becoming a popular solution to massive data analysis in bioinformatics. In order to enable scientists to harness the computing power provided by Cloud platforms, we designed Green Pipe, a scalable computational workflow system, which runs jobs as MapReduce tasks on virtual Hadoop clusters. This paper introduces a power-aware scheduling algorithm in the workflow engine to optimize workflow execution in terms of running time and energy consumption. Experimental results demonstrate the performance improvement in Green Pipe.

Keywords

bioinformatics; cloud computing; data analysis; data flow analysis; power aware computing; workflow management software; GreenPipe; Hadoop based workflow system; MapReduce tasks; bioinformatics; cloud computing; computational workflow system; energy consumption; energy-efficient clouds; massive data analysis; power-aware scheduling algorithm; time consumption; virtual Hadoop clusters; workflow engine; workflow execution optimization; Bioinformatics; Cloud computing; Connectors; Databases; Green products; Proteins; XML; cloud computing; schedule; virtual machine; workflow;

fLanguage

English

Publisher

ieee

Conference_Titel

Parallel and Distributed Processing Symposium Workshops & PhD Forum (IPDPSW), 2012 IEEE 26th International

Conference_Location

Shanghai

Print_ISBN

978-1-4673-0974-5

Type

conf

DOI

10.1109/IPDPSW.2012.273

Filename

6270584