A Comparative Study of Test Coverage-Based Software Reliability Growth Models

Test coverage plays an important role in evaluating the quality of software under test, and serves as a guideline to decide when to stop testing once the desired level of coverage has been achieved. Furthermore, researchers utilized test coverage information by incorporating it in reliability growth modeling to predict software reliability. A number of software reliability growth models that incorporate test coverage has been proposed and developed during the past years. In this paper, we survey, analyze and classify the existing test coverage-based software reliability growth models. We are motivated by the increasing attention paid to incorporate test coverage in reliability modeling, the increasing use of computer systems that requires high reliability and the existence of several test coverage tools. The paper compares the models based on four criteria: (i) metrics used to assess reliability, (ii) coverage criteria used to predict reliability, (iii) the way authors evaluate their model and the evidence they provide to show that the approach achieves its intended goal, and (iv) limitations of the model. Finally, open problems are identified and discussed.