Adjusting software failure rates that are estimated from test data

Software test environments are often different from field environments. Using test data exclusively to estimate a field failure rate will not usually give a very accurate estimate. In this paper, we extend an empirical calibration methodology for adjusting the failure rate estimate obtained from analysing test data. In addition to scaling the estimated failure rate of a fault, we propose scaling the estimated number of residual faults as well. We also derive likelihood ratio tests to formally determine (from previous releases of the software) if test, and field environments are significantly different. We illustrate our new results with two telecommunications case studies. The combination of the likelihood ratio test, and the calibration methodology offers a practical way to extend the application of software reliability growth models to less formal test environments.