DocumentCode :
2371174
Title :
UCBench: A user-centric benchmark suite for C code static analyzers
Author :
Liu, Jiangchao ; Chen, Liqian ; Dong, Longming ; Wang, Ji
Author_Institution :
Sch. of Comput., Nat. Univ. of Defence Technol., Changsha, China
fYear :
2012
fDate :
23-25 March 2012
Firstpage :
230
Lastpage :
237
Abstract :
In recent years, many C code static analyzers, with different abilities of bug detection, have appeared and been applied in various domains. There are so many choices that it becomes hard for programmers to know in detail the strengths as well as limitations of all these analyzers and to find the most suitable ones for their code. In this paper, we propose a benchmark for C code static analyzers, named UCBench, to provide quantitative and qualitative measurements for evaluating analyzers. Being different from other benchmarks, UCBench concentrates more on users´ requirements rather than the improvements of bug detecting technique itself. The major components of UCBench include test case database, evaluation metrics and harness. We classify test cases into several groups according to their attributes and design various user-centric evaluation metrics. Besides, we develop some harness to automate the evaluation process. Finally, we demonstrate our benchmark suite over four C code static analyzers: Flawfinder, Cppcheck, Uno and Splint.
Keywords :
database management systems; program debugging; program diagnostics; software metrics; C code static analyzers; Cppcheck; Flawfinder; Splint; UCBench; Uno; analyzer evaluation; bug detecting technique; evaluation harness; qualitative measurements; quantitative measurements; test case database; user requirements; user-centric benchmark suite; user-centric evaluation metrics; Benchmark testing; Computer bugs; Databases; Libraries; Measurement; Monitoring; Standards;
fLanguage :
English
Publisher :
ieee
Conference_Titel :
Information Science and Technology (ICIST), 2012 International Conference on
Conference_Location :
Hubei
Print_ISBN :
978-1-4577-0343-0
Type :
conf
DOI :
10.1109/ICIST.2012.6221643
Filename :
6221643
Link To Document :
بازگشت