A coverage analysis tool for the effectiveness of software testing

We describe a software testing and analysis tool, called ATAC (Automatic Test Analysis for C), which is developed as a research instrument at Bellcore to measure the effectiveness of testing data. The design, functionality, and usage of ATAC are presented in this paper. Furthermore, to demonstrate the capability and applicability of ATAC, we obtain the 12 program versions of a critical industrial application developed in a recent university/industry N-Version Software project, and use the ATAC tool to analyze and compare coverage of the testing conducted in the program versions. Preliminary results from this investigation show that ATAC could be a powerful testing tool to provide testing metrics and quality control guidance for the certification of high quality software components or systems. It can also assist software reliability researchers and practitioners in searching for the missing link between structure-based testing schemes and software reliability