Power and performance analysis of scalable photonic networks for exascale architecture

As bandwidth demands from high-performance computing (HPC) applications have increased, scalable topology and power-efficiency of network technology are becoming critical parameters for exascale design. Dragonfly topology provides low diameter for exascale networks however, fewer global links reduce the bisection bandwidth while high-radix routers increase the router complexity and area overhead. Further, performance/watt delivered by metallic interconnects significantly increases the power consumed by the network. In this paper, we propose multiple-level topologies that utilize scalable HPC topologies such as k-ary n-cube, flattened butterfly and dragonfly for intra-cabinet and inter-cabinet levels that can lead to higher bisection, manageable radix, and reduced link costs, although at higher packet latency due to increased diameter. As photonic technology offers higher bandwidth density, better power efficiency, and higher performance/watt, our proposed exascale network is designed with photonic transceivers such as vertical-cavity surface emitting lasers (VCSELs) and photodiodes, and complementary-metal-oxide semiconductor (CMOS) routers. Our analytical and simulation studies show that multiple-level topologies can achieve 10-15% more throughput while consuming 40% less power when compared to single-level topologies, and VCSEL-based transceivers can deliver the injection bandwidth of 256 GB/sec/direction while consuming 2-3MW in an exascale systems.