Probabilistic Quality Models to Improve Communication and Actionability

We need to aggregate metric values to make quality assessment actionable. However, due to properties of metric values, i.e., unknown distributions and different measurement scales, they are difficult to aggregate. We present and evaluate a method to aggregate metric values based on observed numerical distributions that are converted into cumulative density functions. We use these to determine the probability of each metric and file, and aggregate these. Our limited study suggests that the method improves correctness, communication, and the ability to take action. However, more evaluation is required.