Provable improvements on branch testing

This paper compares the fault-detecting ability of several software test data adequacy criteria. It has previously been shown that if C₁ properly covers C₂, then C₁ is guaranteed to be better at detecting faults than C₂, in the following sense: a test suite selected by independent random selection of one test case from each subdomain induced by C₁ is at least as likely to detect a fault as a test suite similarly selected using C₂. In contrast, if C₁ subsumes but does not properly cover C₂, this is not necessarily the case. These results are used to compare a number of criteria, including several that have been proposed as stronger alternatives to branch testing. We compare the relative fault-detecting ability of data flow testing, mutation testing, and the condition-coverage techniques, to branch testing, showing that most of the criteria examined are guaranteed to be better than branch testing according to two probabilistic measures. We also show that there are criteria that can sometimes be poorer at detecting faults than substantially less expensive criteria