Classification of Software Defect Detected by Black-Box Testing: An Empirical Study

Software defects which are detected by black box testing (called black-box defect) are very large due to the wide use of black-box testing, but we could not find a defect classification which is specifically applicable to them in existing defect classifications. In this paper, we present a new defect classification scheme named ODC-BD (Orthogonal Defect Classification for Black-box Defect), and we list the detailed values of every attribute in ODC-BD, especially the 300 detailed black-box defect type. We aim to help black-box defect analyzers and black-box testers improve their analysis and testing efficiency. The classification study is based on 1860 black-box defects collected from 39 industry projects and 2 open source projects. Furthermore, two empirical studies are included to validate the use of our ODC-BD. The results show that our ODC-BD can improve the efficiency of black-box testing and black-box defect analysis.