Title :
Assessing and Comparing Vulnerability Detection Tools for Web Services: Benchmarking Approach and Examples
Author :
Antunes, Nuno ; Vieira, Marco
Author_Institution :
Univ. of Coimbra, Coimbra, Portugal
Abstract :
Selecting a vulnerability detection tool is a key problem that is frequently faced by developers of security-critical web services. Research and practice shows that state-of-the-art tools present low effectiveness both in terms of vulnerability coverage and false positive rates. The main problem is that such tools are typically limited in the detection approaches implemented, and are designed for being applied in very concrete scenarios. Thus, using the wrong tool may lead to the deployment of services with undetected vulnerabilities. This paper proposes a benchmarking approach to assess and compare the effectiveness of vulnerability detection tools in web services environments. This approach was used to define two concrete benchmarks for SQL Injection vulnerability detection tools. The first is based on a predefined set of web services, and the second allows the benchmark user to specify the workload that best portrays the specific characteristics of his environment. The two benchmarks are used to assess and compare several widely used tools, including four penetration testers, three static code analyzers, and one anomaly detector. Results show that the benchmarks accurately portray the effectiveness of vulnerability detection tools (in a relative manner) and suggest that the proposed benchmarking approach can be applied in the field.
Keywords :
Web services; program diagnostics; security of data; SQL injection vulnerability detection tools; anomaly detector; benchmarking approach; false positive rates; penetration testers; security-critical Web services; static code analyzers; vulnerability coverage; Benchmark testing; Computer bugs; Measurement; Security; Web services; Benchmarking; and runtime anomaly detection; penetration testing; static analysis; vulnerability detection;
Journal_Title :
Services Computing, IEEE Transactions on
DOI :
10.1109/TSC.2014.2310221