Fast flow classification over Internet

In order to control and manage highly aggregated Internet traffic flows efficiently, we need to be able to categorize flows into distinct classes and to be knowledgeable about their different behaviours. We consider the problem of classifying BGP (border gateway protocol) level prefix flows into a small set of homogeneous classes. We argue that a simple two states hidden Markov model (HMM), even if not sufficient to describe a flow, is sufficient to distinguish between flows and to help in classifying them to homogeneous classes. We propose a classification method based on modelling windows of flow observations using the hidden Markov model and classifying them based on parameters of the model. We use a classical EM algorithm for estimating all model parameters as well as the flow membership probabilities - the probability that a flow belongs to any given class. One of our key contributions is a new and relatively fast method for Internet flow classification. The method is fast as it can classify flows over 30 minutes, i.e. it needs only 6 reports generated at 5 minute intervals (as is usually done for SNMP reports) opening the way for online flow classification and monitoring. We show that our method is powerful in that it is capable of examining macroscopic flows while simultaneously making fine distinctions between different traffic classes. We demonstrate that our scheme can address issues with flows being close to class boundaries and the inherent dynamic behaviour of Internet flows.