Determining the cost impact of SCM system errors

Software Configuration Management (SCM) auditing is the fourth of four sub processes recommended by the IEEE and the ACM in this area. This research is the continuation of ongoing experiments in the use of heuristics for predicting fault rates in systems that support SCM. This paper allocates financial indicators to the business model for a hypothetical Telecommunications company and predicts the potential financial error impact due to Configuration Management errors in the SCM system. This paper focuses on sampling first Use Cases in order to determine the error rates by Operating Profile and then using that knowledge in drawing samples of Test Cases. The 5,388 Test Cases were generated from sources available in open forums and they were injected with 4% of faults; 2.1% carried from Use Cases and 2% added. A total sampling of 492 items was conducted and was able to approximate the financial error rate in 6,006 items at an acceptable level with a 92% reduction in effort. The two stage sampling technique performed better than straight random sampling. When applied to the contribution from each Test Case, random sampling produced above a 6.87% error in the value chain estimate while two stage sampling produced under a 2.72% error in the same estimate.