Effect of Pooled Comparative Information on Judgments of Quality

Quality assessment is the focus of many healthcare initiatives. Yet, it is not well understood how the type of information used in decision support tools to enable judgments of quality based on data impacts the accuracy, consistency, and reliability of judgments made by physicians. Comparative pooled information could allow physicians to judge the quality of their practice by making comparisons with other practices or other specific populations of patients. In this study, resident physicians were provided with varying types of information derived from pooled patient datasets: quality component measures at the individual and group level, a qualitative interpretation of the quality measures using percentile rank, and an aggregate composite quality score. Thirty-two participants viewed 30 quality profiles consisting of information applicable to the practice of 30 deidentified resident physicians. Those provided with quality component measures and a qualitative interpretation of the quality measures (rankings) judged quality of care more similarly to experts and were more internally consistent compared with participants who were provided with quality component measures alone. Reliability between participants was significantly less for those who were provided with a composite quality score compared with those who were not.