An If-While-If Model-Based Performance Evaluation of Ranking Metrics for Spectra-Based Fault Localization

Spectra-based fault localization (SFL) is an automatic fault-localization technique which has received a lot of attention due to its simplicity and effectiveness. SFL uses ranking metric (RM) to rank the risk of fault existence in each program entity after dynamically collecting the necessary information. The evaluation of RMs for SFL has recently become a research focus. To evaluate the average performance of RMs for SFL with different single-fault types, an If-While-If (IWI) model-based approach is presented in this paper. Firstly, through investigating rankings of statements in the IWI model, this paper takes an optimal RM known as an example to analyze its localization effectiveness for five types of single-fault. Secondly, a generic hierarchical method is given in the IWI model to precisely calculate the average performance of RMs. Two experiments, that calculate the average performance of the optimal RM on the IWI model and actual programs, are conducted with five single-fault types. The experimental results agree with theoretical analyses. It is found that the average performance of the optimal RM is related to the number of test cases and the number of program cycles, and the fault type. The IWI model could function as large programs to effectively evaluate RMs for different fault types.