An In-Depth Analysis of TCP and RDMA Performance on Modern Server Platform

Traditional analysis on the TCP performance attributes the overhead of TCP mainly on these aspects: many times of memory access and copy, much data transmission on bus and a large number of CPU cycles required by the protocol processing. Compared to TCP, the main advantage of RDMA lies in its single memory access which brings a higher data transmission bandwidth and lower CPU utilization. However, the hardware platforms that these analyses are based on are already out-of-date from today´s view. The key point is that TCP and RDMA performance can change with the rapid development of the computer architecture and computer hardware components. In this paper, we re-analysis the performance of TCP and RDMA on the modern server platform (e.g., multi-core), and we find that the upper limit of TCP transmission bandwidth has been greatly upgraded compared to the old platform. Besides, the compatibility and complexity problems in programming brought by RDMA technology are still serious. Based on this conclusion, we constructed a network platform which uses multi-thread technology and TCP protocol to achieve a high speed transmission bandwidth in the InfiniBand network environment.