Algorithmic influences on I/O access patterns and parallel file system performance

For many scalable parallel applications, the input/output (I/O) barrier rivals or exceeds that of computation and interprocessor communication. Consequently, scalable parallel secondary and tertiary storage systems are necessary to satisfy the resource demands of many national challenge problems. At present, one major challenge facing the designers of such storage systems is the wide range of I/O access patterns and the lack of general purpose file system policies that achieve high performance for variable I/O requirements. We analyze the I/O behavior of two scientific applications on the Intel Paragon XP/S. Although the two applications solve the same scientific problem and their I/O access patterns are qualitatively similar, their interactions with the file system are decidedly different. Our results show that appropriate tuning of file system policy parameters to I/O demands can significantly increase I/O throughput