Characterization of defects in production systems

In complex production systems, finding the right cause for a perceived failure and exact location of the defect causing this failure has always been a painful exercise. In the past, many attempts have been made to characterize the defect along many dimensions. In this paper we present the results of our empirical analysis of the defect reports received from a live financial trading application. Here we analyze these defects along the three practically important dimensions: defect types, root cause factors and correction mechanism. This analysis helps to identify what makes a failure difficult to reproduce and the corresponding defect difficult to uncover. It will improve defect detection ability as well as it will give directions and pointers for the correction measures to be taken.