An architecture for congestion management in Ethernet clusters

Interconnects for clusters and bladed systems must deliver efficient throughput, low latency, low delay variations and minimal frame drops. The primary technical issues hindering Ethernet adoption for cluster and blade system interconnects are the current methods Ethernet switches use for dealing with congestion, which can happen frequently under cluster and blade system workloads. The common response to congestion is to drop frames and the common method of avoiding the need to drop frames is to utilize very large switch buffers. In this paper, we propose a three-level approach to dealing with congestion that provides efficient throughput, low latency, low delay variations, and can eliminate frame drops, even with very modest sized switch buffers. The approach employs three levels of congestion management: 1) improved link level transient congestion control; 2) oversubscription control at layer 2 subnet ingresses, and 3) end-to-end oversubscription control by the higher layer protocols. We present compelling simulation results showing the incremental benefits provided by each level.