Benchmarks for DDOS Defense Evaluation

There is a critical need for a common evaluation methodology for distributed denial-of-service (DDoS) defenses, to enable their independent evaluation and comparison. We describe our work on developing this methodology, which consists of: (i) a benchmark suite defining the elements necessary to recreate DDoS attack scenarios in a testbed setting, (ii) a set of performance metrics that express a defense system´s effectiveness, cost, and security, and (iii) a specification of a testing methodology that provides guidelines on using benchmarks and summarizing and interpreting performance measures. We identify three basic elements of a test scenario: (i) the attack, (ii) the legitimate traffic, and (iii) the network topology including services and resources. The attack dimension defines the attack type and features, while the legitimate traffic dimension defines the mix of the background traffic that interacts with the attack and may experience a denial-of-service effect. The topology/resource dimension describes the limitations of the victim network that the attack targets or interacts with. It captures the physical topology, and the diversity and locations of important network services. We apply two approaches to develop relevant and comprehensive test scenarios for our benchmark suite: (1) we use a set of automated tools to harvest typical attack, legitimate traffic, and topology samples from the Internet, and (2) we study the effect that select features of the attack, legitimate traffic and topology/resources have on the attack impact and the defense effectiveness, and use this knowledge to automatically generate a comprehensive testing strategy for a given defense