Does the Failing Test Execute a Single or Multiple Faults? An Approach to Classifying Failing Tests

Debugging is an indispensable yet frustrating activity in software development and maintenance. Thus, numerous techniques have been proposed to aid this task. Despite the demonstrated effectiveness and future potential of these techniques, many of them have the unrealistic single-fault failure assumption. To alleviate this problem, we propose a technique that can be used to distinguish failing tests that executed a single fault from those that executed multiple faults in this paper. The technique suitably combines information from (i) a set of fault localization ranked lists, each produced for a certain failing test and (ii) the distance between a failing test and the passing test that most resembles it to achieve this goal. An experiment on 5 real-life medium-sized programs with 18, 920 multiple-fault versions, which are shipped with number of faults ranging from 2 to 8, has been conducted to evaluate the technique. The results indicate that the performance of the technique in terms of evaluation measures precision, recall, and F-measure is promising. In addition, for the identified failing tests that executed a single fault, the technique can also properly cluster them.