Hybrid sampling for estimating flow size distribution and its implementation

Recently, it has been necessary to perform traffic sampling in order to deteriorate the load of capturing and analyzing processes, as the amount of ISP traffic grows. Packet sampling and flow sampling are main sampling techniques. In order to estimate flow size distribution from sampled data, each sampling method has its own advantages and disadvantages. Flow sampling can extract flows in proportion to the original flow size distribution but is difficult to extract large-sized flows due to the heavy-tailed flow size distribution. On the other hand, packet sampling can extract large-sized flows but complete flows cannot be extracted. In this paper, we propose a hybrid sampling method which performs both flow sampling and packet sampling in parallel to utilize both advantages of above two methods and improve estimation accuracy. We also propose cost-effective implementation which employs a general-purpose switch. By verifying with real traffic data, we confirmed the effectiveness of our proposed method in terms of reproducibility.