Characterizing User-Level Network Virtualization: Performance, Overheads and Limits

Connectivity limitations of the current Internet pose challenges to e-Science infrastructure, as communication between resources is of key importance for the sharing of hardware, software and data across geographically distributed institutions. To overcome such limitations many overlay networks have been developed. Existing solutions report performance that can keep up with slow wide-area links - however the question remains as to how overlay networks perform on new high-speed Internet connections. This paper characterizes the performance of user-level packet processing, which is at the core of most overlay networks implementations. To this end, building blocks of packet processing software are identified and characterized, and performance of IP forwarders developed in C and Java are compared through extensive measurements. Factors that affect the performance of overlay networks and limitations of existing solutions are characterized, providing insights on possible improvements. Experimental results show the following: to achieve maximum throughput, only a few microseconds (2 to 5) can be spent in virtual network processing; processing of small packets can limit TCP throughput; and overlays need to be carefully architected as encapsulation overheads can be substantial.