A benchmark evaluation of network intrusion detection systems

This paper outlines a benchmarking method for quantifying the detection sensitivity of network intrusion detection systems (IDS) on an important class of denial-of-service (DOS) attacks using the real-time testbed developed at AFRL. This effort involved the development of tunable attack insertion tools and reference software to measure the state of the victim. Two systems were evaluated, a representative signature-based commercial system and a DARPA research system that relies on statistically based techniques. The experiments tried to answer these questions: Are there regions of operation where the attack tool can degrade performance while escaping detection? Is there any added detection power using the research detection system over standard commercial practice? The results can be summarized as follows: the reference victim´s performance can be degraded without detection; and the research system had a broader detection region than the commercial system