Packet pre-filtering for network intrusion detection

As intrusion detection systems (IDS) utilize more complex syntax to efficiently describe complex attacks, their processing requirements increase rapidly. Hardware and, even more, software platforms face difficulties in keeping up with the computationally intensive IDS tasks, and face overheads that can substantially diminish performance. In this paper we introduce a packet pre-filtering approach as a means to resolve, or at least alleviate, the increasing needs of current and future intrusion detection systems. We observe that it is very rare for a single incoming packet to fully or partially match more than a few tens of IDS rules. We capitalize on this observation selecting a small portion from each IDS rule to be matched in the pre-filtering step. The result of this partial match is a small subset of rules that are candidates for a full match. Given this pruned set of rules that can apply to a packet, a second-stage, full-match engine can sustain higher throughput. We use DefCon traces and recent Snort IDS rule-set, and show that matching the header and up to an 8-character prefix for each payload rule on each incoming packet can determine that on average 1.8 rules may apply on each packet, while the maximum number of rules to be checked across all packets is 32. Effectively, packet pre-filtering prevents matching at least 99% of the SNORT rules per packet and as a result minimizes processing and improves the scalability of the system. We also propose and evaluate the cost and performance of a reconfigurable architecture that uses multiple processing engines in order to exploit the benefits of pre-filtering.