On Building Scientific Workflow Systems for Data Management in the Cloud

Author

Simmhan, Yogesh ; Barga, Roger ; van Ingen, C. ; Lazowska, Ed ; Szalay, Alex

Author_Institution

Microsoft Res., San Francisco, CA

fYear

2008

fDate

7-12 Dec. 2008

Firstpage

434

Lastpage

435

Abstract

Scientific workflows have become an archetype to model in silico experiments in the Cloud by scientists. There is a class of workflows that are used to by "data valets" to prepare raw data from scientific instruments into a science-ready form for use by scientists. These share data-intensive traits with traditional scientific workflows, yet differ significantly, for example, in the required degree of reliability and the type of provenance collected. We compare and contrast science application and data valet workflows through exemplar eScience projects to drive shared and unique requirements for scientific workflows across diverse users in a Science Cloud.

Keywords

data analysis; scheduling; scientific information systems; workflow management software; data management; data valet workflow; data-intensive trait sharing; distributed resources scheduling; e-science project; science cloud computing; scientific data analysis; scientific instrument; scientific workflow system; Cloud computing; Collaborative work; Data analysis; Grid computing; Instruments; Object detection; Pipelines; Relational databases; Telescopes; USA Councils; cloud computing; data management; eScience; scientific workflow frameworks;

fLanguage

English

Publisher

ieee

Conference_Titel

eScience, 2008. eScience '08. IEEE Fourth International Conference on

Conference_Location

Indianapolis, IN

Print_ISBN

978-1-4244-3380-3

Electronic_ISBN

978-0-7695-3535-7

Type

conf

DOI

10.1109/eScience.2008.150

Filename

4736833