Dynamic I/O congestion control in scalable lustre file system

Author

Yingjin Qian ; Ruihai Yi ; Yimo Du ; Nong Xiao ; Shiyao Jin

Author_Institution

Marine Tracking & Control Dept., China Satellite, Jiangyin, China

fYear

2013

fDate

6-10 May 2013

Firstpage

1

Lastpage

5

Abstract

This paper introduces a scalable I/O model of Lustre file system and propose a dynamic I/O congestion control mechanism to support the incoming exascale HPC systems. Under its control, clients are allowed to issue more concurrent I/O requests to servers, which optimizes the utilization of the network/server resources and improves the I/O throughput, when servers are under light load; on the other hand, it can throttle the clients´ I/O and limit the number of I/O requests queued on the server to control the I/O latency and avoid congestive collapse, when the server is overloaded. The results of series of experiments demonstrate the effectiveness of our congestion control mechanism. It prevents the occurrence of congestive collapse and on this premise it can maximize the I/O throughput for the scalable Lustre file system.

Keywords

file organisation; parallel processing; I/O latency; I/O throughput; congestive collapse; dynamic I/O congestion control; exascale HPC system; network-server resources; scalable I/O model; scalable Lustre file system; Adaptation models; Bandwidth; Concurrent computing; File systems; Servers; Throughput; Time measurement; HPC; I/O congestion control; I/O intensive; Lustre; QoS; clustered file system; response time; scalability;

fLanguage

English

Publisher

ieee

Conference_Titel

Mass Storage Systems and Technologies (MSST), 2013 IEEE 29th Symposium on

Conference_Location

Long Beach, CA

ISSN

2160-195X

Print_ISBN

978-1-4799-0217-0

Type

conf

DOI

10.1109/MSST.2013.6558432

Filename

6558432