Efficient Pattern Based I/O Analysis of Parallel Programs

Modern HPC systems are constructed by placing more and more cores in a single machine. To utilize this kind of machines efficiently, many parallel processes have to be used. The performance analysis of massively parallel program runs becomes more and more complicated as the number of events that are generated while the program is traced grows linearly with the number of processes. In order to utilize large HPC systems efficiently, parallel applications have to execute I/O requests in parallel. Analyzing these I/O requests and optimizing this part of a parallel program requires a deep knowledge of all issued requests and the dependencies between these requests. Traditional tracing facilities record all necessary information, including all synchronization events. We present a novel approach to reduce the amount of information needed for an I/O analysis in program traces significantly. This reduction enables a further analysis of the reduced data set in other tools that for example detect request patterns. Our approach is based on a specialized graph that is constructed from an event trace. This paper describes a systematic methodology to reduce the initial graph by merging adjacent vertices. As an extension we also describe how this merging step can be combined with the graph construction which significantly reduces the runtime of the algorithm in practice. An example that demonstrates the practical application of the methodology to real world use cases concludes the paper. After applying the reduction operation to application traces in the example the amount of synchronization events remaining is in the order of the number of I/O events.