Title :
HAS: Heterogeneity-Aware Selective Data Layout Scheme for Parallel File Systems on Hybrid Servers
Author :
Shuibing He ; Xian-He Sun ; Haider, Adnan
Author_Institution :
Sch. of Comput., Wuhan Univ., Wuhan, China
Abstract :
Hybrid parallel file systems (PFS), consisting of multiple HDD and SSD I/O servers, provide a promising design for data intensive applications. The efficiency of a hybrid PFS relies on the file´s data layout. However, most current layout strategies are designed and optimized for homogeneous servers. Using them directly in a hybrid PFS neither addresses the heterogeneity of servers nor the varying access patterns of applications, making hybrid PFSs disappointingly inefficient. In this paper, we propose HAS, a novel heterogeneity-aware selective data layout scheme for hybrid PFSs. HAS alleviates the inter-server load imbalance through skewing data distribution on heterogeneous servers based on their storage performance. To largely improve the entire system´s I/O efficiency, HAS adaptively selects the optimal data layout from three typical candidates according to the application´s data access patterns, based on a newly developed selection and distribution algorithm. We have implemented HAS within OrangeFS to provide efficient data distribution for data-intensive applications. Our extensive experiments validate that HAS significantly increases the I/O throughput of hybrid PFSs, compared to existing data layout optimization methods.
Keywords :
disc drives; parallel processing; storage management; HAS; I/O efficiency; PFS; SSD I/O server; data intensive application; heterogeneity-aware selective data layout inter-server load imbalance; heterogeneity-aware selective data layout scheme; heterogeneous server; homogeneous server; hybrid parallel file system; hybrid server; layout strategy; multiple HDD server; optimal data layout; skewing data distribution; storage performance; varying access pattern; Analytical models; Concurrent computing; Data models; Data transfer; Layout; Optimization; Servers; Data Layout; Parallel File system; Parallel I/O System; Solid State Drive;
Conference_Titel :
Parallel and Distributed Processing Symposium (IPDPS), 2015 IEEE International
Conference_Location :
Hyderabad
DOI :
10.1109/IPDPS.2015.23