Adaptive aggregation flow measurement on high speed links

While network traffic may be characterized by many different criteria, it is ease to aggregate traffic along one dimension at a time. Unfortunately, by aggregating traffic along any single dimension, the network manager inevitably loses some interesting information. While the network manager can expose this structure by using finer grained representations, such as flows, he then must manage the excessive detail contained in such a representation. We define our traffic clusters in terms of the five fields typically used to define a fine-grained flow: source IP address, destination IP address, protocol, source port and destination port. Unlike others flow monitoring methods, such as NetFlow and ANF, we only keep the heavy-tailed flows and sampled short flows on a non-uniform sampling method with the flow length. The aggregation traffic can be estimated by these sampled flows and can keep the estimated accuracy at the same time. Experiment studies show our approach can significantly improve both the accuracy and efficiency in network aggregation flow monitoring comparing to other existing approaches.