Composite Release Values for Normalized Product-level Metrics

For metrics normalized using field data, variability is often high, since different classes of customers use different features of large releases. It is important to understand the quality health of the entire release, so we need to have a way to estimate the overall normalized metric value for the entire release, across all product lines. This paper looks at three different ways to calculate release values for a key normalized metric, software defects per million usage hours per month (SWDPMH). The ´Aggregate,´ ´Averaging,´ and ´Indexing´ approaches are defined and examined for two major IOS release variants. Each of these approaches has general strengths and weaknesses, and these are described. Each of the three approaches has been found to be useful in particular situations, and these scenarios are also described. The primary objective of this study is to find an accurate method to estimate SWDPMH for software releases that can be used to improve release management operations, corporate goaling, and best practices evaluation for successor feature releases. Since the methods described are not particular to SWDPMH, we believe they may be useful for other normalized metrics-this is an area of future work.