Exploiting transparent remote memory access for non-contiguous- and one-sided-communication

The availability of an implementation of the Message Passing Interface (MPI) is essential for each interconnect designed for communication in HPC clusters. Using the open-source implementation of MPI, MPICH, and creating a communication device for it based upon the low-level communication libraries of the interconnect, this goal can be achieved quiet easily. However, optimizing the resulting MPI implementation to make maximum benefit of the characteristics of the interconnect is a more complex task. This paper presents two of the most recent optimizations in SCI-MPICH, an MPICH variant for the SCI interconnect, which make use of the global shared memory provided by this interconnect: efficient communication with non-contiguous MPI datatypes and one-sided communication according to the MPI-2 standard. We show that the transparent low-latency communication characteristics of SCI provides these techniques an excellent platform.