Extracting baseline patterns in Internet traffic using Robust Principal Components

Robust BaseLine (RBL) is a formal technique for extracting the baseline of network traffic to capture the underlying traffic trend. A range of applications such as anomaly detection and load balancing rely on baseline estimation. Once the fundamental period of the pattern for analysis is recognized, e.g., based on user interest or a period detector such as Autocorrelation Function (ACF), the basic extraction is carried out in two steps. First, the common component across the dataset is separated using Robust Principal Component Analysis (RPCA). The fundamental pattern in the common component is extracted using Principal Component Analysis (PCA) in the second step. Scaling factors required to fit the base-pattern back into the data are returned automatically by PCA. Two types of traffic baselines may be extracted: RBL-L captures the common behavior across time on a single link, and RBL-N captures the common behavior across a network of links, i.e., in space. RBL-N is particularly useful for specifying traffic matrices more efficiently over time, which normally requires multiple updates to follow baseline trends. The derived base-patterns for a single link or a single time period is then extended over the entire network or thru the entire observation period with a compressive analysis. The compressed base-pattern provides a smoother baseline and also a filter to separate baseline traffic and the deviations on the fly from traffic measurements. When compared against BLGBA (Baseline for Automatic Backbone Management) the proposed scheme provides a less noisy, more precisely fitting baseline. It is also more effective in revealing anomalies.