Title :
Evaluating I/O characteristics and methods for storing structured scientific data
Author :
Ching, Avery ; Choudhary, Alok ; Liao, Wei-keng ; Ward, Lee ; Pundit, Neil
Author_Institution :
Dept. of EECS, Northwestern Univ., Evanston, IL
Abstract :
Many large-scale scientific simulations generate large, structured multi-dimensional datasets. Data is stored at various intervals on high performance I/O storage systems for checkpointing, post-processing, and visualization. Data storage is very I/O intensive and can dominate the overall running time of an application, depending on the characteristics of the I/O access pattern. Our NCIO benchmark determines how I/O characteristics greatly affect performance (up to 2 orders of magnitude) and provides scientific application developers with guidelines for improvement. In this paper, we examine the impact of various I/O parameters and methods when using the MPI-IO interface to store structured scientific data in an optimized parallel file system
Keywords :
file organisation; input-output programs; parallel processing; I/O access pattern; checkpointing; high performance I/O storage systems; large-scale scientific simulations; optimized parallel file system; structured scientific data storage; Adaptive mesh refinement; Character generation; Computational modeling; Concurrent computing; Data visualization; File systems; Guidelines; Laboratories; Large-scale systems; Testing;
Conference_Titel :
Parallel and Distributed Processing Symposium, 2006. IPDPS 2006. 20th International
Conference_Location :
Rhodes Island
Print_ISBN :
1-4244-0054-6
DOI :
10.1109/IPDPS.2006.1639306
