IDS Alert Correlation in the Wild With EDGe

Intrusion detection systems (IDSs) produce a large number of alerts, which overwhelm their operators, e.g., a deployment of the popular Snort IDS in the campus network of ETH Zurich (which includes more than 40 thousand hosts) produces on average 3 million alerts per day. In this paper, we introduce an IDS alert correlator, which we call Extrusion Detection Guard (EDGe), to detect infected hosts within a monitored network from IDS alerts. EDGe detects several malware that exhibit a multi-stage behavior and it can identify the family and even variant of certain malware, which helps to remediate and prioritize incidents. Our validation based on manual real-time analysis of a sample of detected incidents shows that only 15% of the detected infections are false positives. In addition, we compare EDGe with a state-of-the-art previous work and show that EDGe finds 60% more infections and has a lower number of false positives. A large part of this paper focuses on characterizing 4,358 infections (13.4 new infections per day) detected with EDGe from a unique dataset of 832 million IDS alerts collected from an operational network over a period of 9 months. Our characterization shows that infections exhibit spatial correlations and attract many further inbound attacks. Moreover, we investigate attack heavy hitters and show that client infections are significantly more bursty compared to server infections. Finally, we compare the alerts produced by different malware families and highlight key differences in their volume, aliveness, fanout, and severity.