BurstMem: A high-performance burst buffer system for scientific applications

Author

Teng Wang ; Oral, Sarp ; Yandong Wang ; Settlemyer, Brad ; Atchley, Scott ; Weikuan Yu

Author_Institution

Auburn Univ., Auburn, AL, USA

fYear

2014

fDate

27-30 Oct. 2014

Firstpage

71

Lastpage

79

Abstract

The growth of computing power on large-scale systems requires commensurate high-bandwidth I/O systems. Many parallel file systems are designed to provide fast sustainable I/O in response to applications´ soaring requirements. To meet this need, a novel system is imperative to temporarily buffer the bursty I/O and gradually flush datasets to long-term parallel file systems. In this paper, we introduce the design of BurstMem, a high-performance burst buffer system. BurstMem provides a storage framework with efficient storage and communication management strategies. Our experiments demonstrate that BurstMem is able to speed up the I/O performance of scientific applications by up to 8.5× on leadership computer systems.

Keywords

buffer storage; natural sciences computing; parallel processing; BurstMem; communication management strategies; high bandwidth I/O systems; high performance burst buffer system; large scale systems; leadership computer systems; parallel file systems; scientific applications; storage framework; Bandwidth; Checkpointing; Indexing; Message systems; Random access memory; Servers; Storage management;

fLanguage

English

Publisher

ieee

Conference_Titel

Big Data (Big Data), 2014 IEEE International Conference on

Conference_Location

Washington, DC

Type

conf

DOI

10.1109/BigData.2014.7004215

Filename

7004215