Performance Measurement of an Integrated NIC Architecture with 10GbE

The deployment of 10 Gigabit Ethernet (10 GbE) connections to servers has been hampered by the "fast-network-slow-host" phenomenon. Recently, the integration of network interfaces (INICs) is proposed to tackle the performance mismatch. While significant advantages over PCI-based discrete NICs (DNICs) were shown in prior work using simulation methodologies, it is still unclear how INICs perform on real machines with 10 GbE. This paper is the first to study the impact of INICs by extensive evaluations through micro-benchmarks on a highly threaded Sun Niagara 2 processor. The processor is the industrypsilas first "system on a chip," integrating two 10 GbE NICs. We observe that the INIC only shows its advantage over the DNIC with large I/O sizes. It improves 7.5% network bandwidth while saving 20% relative CPU utilization. We characterize the system behaviors to fully understand the performance benefits with respect to different number of connections, OS overhead, instruction counts, and cache misses etc. All of our studies reveal that there is a benefit of integrating NICs onto CPUs, but the gain is somewhat marginal. More aggressive integrated NIC designs should be adopted for higher speed networks like the upcoming 40 GbE and 100 GbE.