Does Software Process Improvement Reduce the Severity of Defects? A Longitudinal Field Study

As firms increasingly rely on information systems to perform critical functions, the consequences of software defects can be catastrophic. Although the software engineering literature suggests that software process improvement can help to reduce software defects, the actual evidence is equivocal. For example, improved development processes may only remove the “easier” syntactical defects, while the more critical defects remain. Rigorous empirical analyses of these relationships have been very difficult to conduct due to the difficulties in collecting the appropriate data on real systems from industrial organizations. This field study analyzes a detailed data set consisting of 7,545 software defects that were collected on software projects completed at a major software firm. Our analyses reveal that higher levels of software process improvement significantly reduce the likelihood of high severity defects. In addition, we find that higher levels of process improvement are even more beneficial in reducing severe defects when the system developed is large or complex, but are less beneficial in development when requirements are ambiguous, unclear, or incomplete. Our findings reveal the benefits and limitations of software process improvement for the removal of severe defects and suggest where investments in improving development processes may have their greatest effects.