Classifying faulty modules with an extension of the H-index

Background. Fault-proneness estimation models provide an estimate of the probability that a software module is faulty. These models can be used to classify modules as faulty or non-faulty, by using a fault-proneness threshold: modules whose fault-proneness exceeds the threshold are classified as faulty and the others as non-faulty. However, the selection of the threshold value is to some extent subjective, and different threshold values may lead to very different results in terms of classification accuracy. Objective. We introduce and empirically validate a new approach to setting thresholds, based on an extension of the H-index defined in Bibliometrics, called the Fault-proneness H-Index. We define and use this extension to identify the most fault-prone modules, which are candidates for intensive Verification & Validation activities. Method. We carried out the empirical validation on two data sets with different faultiness characteristics hosted on the PROMISE repository, by using T-times K-fold cross validation. We computed Precision, Recall, the F - measure, and a weighted version of the F - measure for the results obtained with our approach and compared them with the values obtained with other approaches based on several thresholds. Results. In the empirical validation, our approach provides better classification results than those based on most other thresholds, according to some classification accuracy indicators, in a statistically significant way. Conclusions. Our approach seems to be able to contribute to accurately classifying modules as faulty or non-faulty.