Using Cluster Analysis to Identify Coincidental Correctness in Fault Localization

In order to improve efficiency of debugging, many fault localization techniques have been proposed to find out the program entities that are likely to contain faults. However, recent researches indicate that the effectiveness of fault localization techniques suffers from occurrences of coincidental correctness, which means execution result of test cases that exercise faulty statements indicate no failure information. This paper presents a strategy using cluster analysis to identify coincidental correctness in test sets for fault localization. Test cases that exercise same faulty statements are expected to be grouped together by cluster analysis, and then during debugging these tests that are identified to contain coincidental correctness can be used to improve effectiveness of fault localization techniques. To evaluate our technique, we conducted an experiment on some Siemens Suit programs. The experimental results show that the strategy is effective at automatically identifying coincidental correct tests.