Practical anomaly detection based on classifying frequent traffic patterns

Detecting network traffic anomalies is crucial for network operators as it helps to identify security incidents and to monitor the availability of networked services. Although anomaly detection has received significant attention in the literature, the automatic classification of network anomalies still remains an open problem. In this paper, we introduce a novel scheme and build a system to detect and classify anomalies that is based on an elegant combination of frequent item-set mining with decision tree learning. Our approach has two key features: 1) effectiveness, it has a very low false-positive rate; and 2) simplicity, an operator can easily comprehend how our detector and classifier operates. We evaluate our scheme using traffic traces from two real networks, namely from the European-wide backbone network of GEÁNT and from a regional peering link in Spain. In both cases, we achieve an overall classification accuracy greater than 98% and a false-positive rate of approximately only 1%. In addition, we show that it is possible to train our classifier with data from one network and use it to effectively classify anomalies in a different network. Finally, we have built a corresponding anomaly detection and classification system and have deployed it as part of an operational platform, where it is successfully used to monitor two 10Gb/s peering links between the Catalan and the Spanish national research and education networks (NREN).