An application based MPI message throughput benchmark

Recent trends in high performance computing have renewed interest in the ability of platforms to sustain high message throughput rates. The continued growth in platform scale, combined with emerging application areas, are pushing platforms to support increasing message rates. Best-case message throughput has grown in previous hardware generations due to growing clock rates and software optimization techniques. However, previous work has shown that MPI receive queue length and cache hit rates can drastically impact message throughput, leading to a significantly lower worst-case message throughput. This paper introduces the Sandia message throughput benchmark which measures message throughput using a communication pattern which is neither best-case nor worst-case, but which mimics communication patterns found in real-world applications. Results on InfiniBand, Myrinet, and Cray XT platforms are presented, and suggest that message rates on some networks are greatly impacted by cache invalidation between communication phases, simultaneously sending and receiving, and by communicating with more than one peer simultaneously.