Classification and Discovery of Rule Misconfigurations in Intrusion Detection and Response Devices

The signature-based intrusion detection is one of the most commonly used techniques implemented in modern intrusion detection systems (IDS). Being based on a set of rules, i.e., attack signatures, the accuracy and reliability of IDS detection heavily depend on the quality of the employed rule set. In this context, any conflicts that arise between rules create ambiguity in classification of network traffic or host events, not only affecting the performance of IDS, but also putting the system in a vulnerable position. Currently existing techniques for conflict detection focus primarily on the security policy of the network devices: IPSec, routers, firewalls. In this paper we address the conflict detection in host and network-based intrusion detection and response devices and present a rule management framework that allows rule set analysis for potential conflicts. We demonstrate the advantages of the proposed approach on three collections of attack signatures: the set provided by the vendor of the commercial IDS and the rule sets of the open source Snort IDS and bleeding edge threats. Our analysis reveal conflicts in each of them.