A multi-level approach for understanding I/O activity in HPC applications

I/O has become one of the determining factors of HPC application performance. Understanding an application´s I/O activity requires a multi-level view of the I/O function flow that includes high-level I/O libraries. We have developed a tracing framework, called Recorder, that captures I/O function calls at multiple layers of the parallel I/O stack without requiring source code modifications. In this paper, we show how Recorder´s trace output can be used to investigate I/O activity and identify performance inefficiencies in two I/O benchmarks running on a leading edge HPC platform. Future work to organize and present the collected information more intuitively will further increase the value of Recorder´s capabilities. We believe that a multi-level I/O tracing framework can provide key insights to end users and I/O library developers working to improve I/O on HPC platforms.