Detection of anomalous network packets using lightweight stateless payload inspection

A real-time packet-level anomaly detection approach for high-speed network intrusion prevention is described. The approach is suitable for small and fast hardware implementation and was designed to be embedded in network appliances. Each network packet is characterized using a novel technique that efficiently maps the payload histogram onto a simple pair of features using hypercube hash functions, which were chosen for their implementation efficiency in both hardware and software. This two-dimensional feature space is quantized into a binary bitmap representing the normal and anomalous feature regions. The potential loss of accuracy due to the reduction in feature space is countered by the ability of the bitmaps to capture nearly arbitrary shaped regions in the feature space. These bitmaps are used as the classifiers for real-time detection. The proposed method is extremely efficient in both the offline machine learning and real-time detection components. Results using the 1999 DARPA Intrusion Detection Evaluation Data Set yield a 100% detection of all applicable attacks, with extremely low false positive rate. The approach is also evaluated on real traffic captures.