Effective Change Detection in Large Repositories of Unsolicited Traffic

When monitoring unsolicited network traffic automated detection and characterization of abrupt changes in the traffics statistical properties is important. These abrupt changes can either be due to a single or multiple anomalous activities taking place at the same time. The start of a new anomalous activity while another anomalous activity is in operation will result in a new change nested within the previous change. Although detection of abrupt changes to identify malicious activities has received considerable attention in the past, automated detection of nested changes has not been addressed. In this paper a dynamic sliding window cumulative sum (CUSUM) algorithm is proposed to automatically identify these nested changes. The novelty of the proposed technique lies in its ability to automatically detect nested changes, without which interesting activities may go undetected, and its effectiveness in identifying both the start and the end of the individual changes. Using an analysis of real network traces, we show that the identified nested changes were indeed due to distinct malicious behaviours taking place in parallel.