Modeling the Diagnostic Efficiency of Regression Test Suites

Diagnostic performance, measured in terms of the manual effort developers have to spend after faults are detected, is not the only important quality of a diagnosis. Efficiency, i.e., the number of tests and the rate of convergence to the final diagnosis is a very important quality of a diagnosis as well. In this paper we present an analytical model and a simulation model to predict the diagnostic efficiency of test suites when prioritized with the information gain algorithm. We show that, besides the size of the system itself, an optimal coverage density and uniform coverage distribution are needed to achieve an efficient diagnosis. Our models allow us to decide whether using IG with our current test suite will provide a good diagnostic efficiency, and enable us to define criteria for the generation or improvement of test suites.