Testing the Value of Checklists in Code Inspections

Checklists are an important part of code and design inspections. Ideally, they aim to increase the number of faults found per inspection hour by highlighting known areas of previous failure. In practice, although some researchers have quantified checklists´ benefits, the conclusions´ statistical robustness hasn´t been as well represented. The author subjects checklists´ effectiveness to formal statistical testing, using data from 308 inspections by industrial engineers over a three-year period. The results showed no evidence that checklists significantly improved these inspections. Further analysis revealed that individual inspection performance varied by a factor of 10 in terms of faults found per unit time, and individuals found on average about 53 percent of the faults. Two-person teams found on average 76 percent of the faults.