Incremental Estimation of Project Failure Risk with Naive Bayes Classifier

Background: Estimation and prediction techniques using quantitative models are considered to be major contributors to early risk control of software projects. Since software projects tend to involve instability and uncertainty, "dynamic" approaches, which perform early estimations and predictions with limited data and then update them incrementally with newly acquired data during project execution, are highly effective. Aim: showing the effectiveness of the incremental estimation of project failure risk using Naïve Bayes classifier. Method: We conducted experiments with data of 104 projects from an organization in which the prediction results obtained using Naïve Bayes classifier were compared with those obtained using the Poisson regression model in each development phase: low-level design (LD), coding (CD), and unit testing (UT). The experiments were carried out with 10-fold cross-validation and the results were evaluated with the area under ROC curve (AUC). Results: Whereas the AUCs obtained using Poisson regression were 0.708, 0.709, and 0.663, those obtained using Naïve Bayes classifier were 0.702, 0.748, and 0.764, respectively, in LD, CD, and UT. Conclusions: The results of the experiments in which Naïve Bayes classifier achieved overall higher accuracy and robustness than Poisson regression support the effectiveness of applying Naïve Bayes classifier to the incremental estimation of project failure risk.