Low-latency software defined network for high performance clouds

Multi-tenant clouds with resource virtualization offer elasticity of resources and elimination of initial cluster setup cost and time for applications. However, poor network performance, performance variation and noisy neighbors are some of the challenges for execution of high performance applications on public clouds. Utilizing these virtualized resources for scientific applications, which have complex communication patterns, require low latency communication mechanisms and rich set of communication constructs. To minimize the virtualization overhead, a novel approach for low latency network for HPC Clouds is proposed and implemented over a multi-technology software defined network. The efficiency of the proposed low-latency Software Defined Networking is analyzed and evaluated for high performance applications. The results of the experiments show that the latest Mellanox FDR InfiniBand interconnect and Mellanox OpenStack plugin gives the best performance for implementing VM-based high performance clouds with large message sizes.