Developing of the Appropriateness Evaluation Protocol for Public Hospitals in Iran

Background: Employment of utilization review instruments is a method for managing costs and efficiency in the healthcare systems. Objectives: This study developed an instrument for measuring the level of inappropriate acute hospital admissions and days of care in Iran. Patients and Methods: The American version of the Appropriateness Evaluation Protocol (AEP) was modified, using the agreement method, by a multidisciplinary group of physicians. We conducted a retrospective descriptive study of 273 randomly selected patients admitted to Imam Khomeini Hospital of Tehran University of Medical Sciences in Tehran, Iran. For the reliability study, two nurses were asked to review patients’ medical records using the instrument. Validity was appraised by pairs of clinicians, including two general surgeons, two internists and two gynecologists. The degree of consensus between the three pairs of clinicians was compared with that of the nurses. Results: Inter-rater and intra-rater reliability testing revealed an excellent level of consensus between the two nurses employing the AEP in all the studied departments. Overall agreement was > 92%, while the specific appropriate agreement and specific inappropriate agreement were > 88% and > 83%, respectively. External validity testing of the instrument yielded a sensitivity > 0.785, specificity > 0.55, and positive and negative predictive values > 0.775 and > 0.555, respectively. The kappa statistic for the nurses who applied the AEP and clinicians using personal judgment were perfect (k > 0.85) and substantial (k > 0.68), respectively. Conclusions: The results illustrate that the Iranian version of the AEP (IR-AEP) could be a reliable and valid instrument for assessing the level of inappropriate acute hospital admissions and days of care in the Iranian context