Exploring Infiniband Hardware Virtualization in OpenNebula towards Efficient High-Performance Computing

It has been widely accepted that software virtualization has a big negative impact on high-performance computing (HPC) application performance. This work explores the potential use of Infiniband hardware virtualization in an Open Nebula cloud towards the efficient support of MPI-based workloads. We have implemented, deployed, and tested an Infiniband network on the Fermi Cloud private Infrastructure-as-a-Service (IaaS) cloud. To avoid software virtualization towards minimizing the virtualization overhead, we employed a technique called Single Root Input/Output Virtualization (SRIOV). Our solution spanned modifications to the Linux´s Hypervisor as well as the Open Nebula manager. We evaluated the performance of the hardware virtualization on up to 56 virtual machines connected by up to 8 DDR Infiniband network links, with micro-benchmarks (latency and bandwidth) as well as with a MPI-intensive application (the HPL Linpack benchmark).