An analysis of signature overlaps in Intrusion Detection Systems

An Intrusion Detection System (IDS) protects computer networks against attacks and intrusions, in combination with firewalls and anti-virus systems. One class of IDS is called signature-based network IDSs, as they monitor network traffic, looking for evidence of malicious behaviour as specified in attack descriptions (referred to as signatures).Many studies report that IDSs, including signature-based network IDSs, have problems to accurately identify attacks. One possible reason that we observed in our past work, and that is worth investigating further, is that several signatures (i.e., several alarms) can be triggered on the same group of packets, a situation we coined overlapping signatures. This paper presents a technique to precisely and systemat ically quantify the signature overlapping problem of an IDS signature database. The solution we describe is based on set theory and finite state automaton theory, and we experiment with our technique on one widely-used and maintained IDS. Results show that our approach is effective at systematically quantifying the overlap problem in one IDS signature database, and can be potentially used on other IDSs.