W-VST: A Testbed for Evaluating Web Vulnerability Scanner

In the recent years, web applications have become increasingly popular for delivering security critical services. Because web applications are exposed to various threats and attacks, numerous tools, including commercial tools and open source software, have been developed for detecting web application vulnerabilities, called web vulnerability scanner. Many studies have focused on evaluating web vulnerability scanners by comparing the vulnerability coverage, precision, recall, and time complexity. However, tremendous new attack scenarios and various hacking techniques usually cause erroneous judgement by the scanners and a comprehensive scan often results in redundant vulnerability alerts. Therefore, an efficient detection tools is essential and can be extremely helpful to the users. In this paper, we propose the advanced confusion matrix to estimate the performance of Web vulnerability scanners and then propose a cost-effective approach with three main phases to evaluating vulnerability scanners by additionally considering the reduction of redundant vulnerability alert. We define the redundant alert problem in scanner evaluation based upon two attributes, true duplication (TD) and false duplication (FD). Accordingly, we build up the Web Vulnerability Scanner Testbed, W-VST. Two experiments have been made to evaluate the performance. The experimental results indicate that our evaluation approach can verify the performance of scanners and W-VST is efficient in tool evaluation.