A Trace Driven Comparison of Latency Hiding Techniques for Network Processors

Caching, multithreading and the combination of them are the major latency hiding techniques adopted in network processors (NPs). Although they achieve great success in general purpose processors (GPPs), none of them have been well studied under the new context of packet processing. In this paper, we simulate the processing procedure of a four-PE (processing element) network processor and thoroughly evaluate different configurations of these techniques with real-life packet traces. Our major findings include: (1) In general, all of these latency hiding techniques effectively increase the traffic throughput and robustness of NP; but thread allocation policy has great impact on their performance. (2) If assigning packets of the same flow to different threads is allowed, multithreading keeps the PE in a working state as long as possible and less jitter in packet sending rate is resulted than caching schemes; otherwise, a cache with a reasonable size outperforms multithreading in almost all metrics such as traffic throughput, packet loss rate, queuing and total delay. (3) When access latency is comparable to the working time of execution unit, the performance of multithreading is more sensitive to packet arrival process and memory reference pattern than caching. In short, caching and multithreading have their respective advantages under different environment. In some cases, combined caching and multithreading tend to bring more performance gain than simply adding more threads or cache entries.