Design and implementation of virtual memory-mapped communication on Myrinet

Describes the design and implementation of the Virtual Memory-Mapped Communication (VMMC) model on a Myrinet network of PCI-based PCs. VMMC has been designed and implemented for the SHRIMP multicomputer, where it delivers user-to-user latency and bandwidth close to the limits imposed by the underlying hardware. The goal of this work is: to provide an implementation of VMMC on a commercially available hardware platform; to determine whether the benefits of VMMC can be realized on the new hardware; and to investigate network interface design tradeoffs by comparing SHRIMP with Myrinet and its respective VMMC implementation. Our Myrinet implementation of VMMC achieves 9.8 μs one-way latency and provides 108.4 MByte/s user-to-user bandwidth. Compared to SHRIMP, the Myrinet implementation of VMMC incurs relatively higher overhead and demands more network interface resources (LANai processor, on-board SRAM) but requires less operating system support