Do you trust your software-based traffic generator?

Networking research often relies on synthetic traffic generation in its experimental activities; from generation of realistic workload to active measurements. Often researchers adopt software-based generators because of their flexibility. However, despite the increasing number of features (e.g., replication of complex traffic models), they are still suffering problems that can undermine the correctness of experiments: what is generated is sometimes far from what is requested by the operator. In this article, by analyzing four of the most used packet-level traffic generators in literature, we show how they fail to follow the requested profiles. Moreover, we identify and discuss key concepts affecting their accuracy as well as mechanisms commonly adopted to improve it. This contribution goes toward improving the knowledge researchers and practitioners should have of the tools used in experimental works, and at the same time illustrates some directions for the use and design of software-based traffic generators.