Peer-to-Peer Data Sharing for Scientific Workflows on Amazon EC2

Author

Agarwal, Rohit ; Juve, Gideon ; Deelman, Ewa

Author_Institution

Dept. of Comput. Sci. & Eng., Indian Inst. of Technol., Ropar, Ropar, India

fYear

2012

fDate

10-16 Nov. 2012

Firstpage

82

Lastpage

89

Abstract

In this paper, we consider the problem of data sharing in scientific workflows running on the cloud. We present the design and evaluation of a peer-to-peer approach to help solve this problem. We compare the performance of our peer-to-peer file manager with that of two network file systems for storing data for a typical data-intensive workflow application. Our results show that while our peer-to-peer file manager performs significantly better than one of the network file systems tested, it does not perform as well as the other. Finally, we discuss the various issues that might have affected the performance of our peer-to-peer file manager.

Keywords

cloud computing; natural sciences computing; peer-to-peer computing; software performance evaluation; storage management; Amazon EC2; cloud computing; data storage; data-intensive workflow application; network file systems; peer-to-peer data sharing; peer-to-peer file manager performance; scientific workflows;

fLanguage

English

Publisher

ieee

Conference_Titel

High Performance Computing, Networking, Storage and Analysis (SCC), 2012 SC Companion:

Conference_Location

Salt Lake City, UT

Print_ISBN

978-1-4673-6218-4

Type

conf

DOI

10.1109/SC.Companion.2012.23

Filename

6495805