Recalibration of the pediatric risk of admission score using a multi-institutional sample

Study objective Case-mix adjustment is a critical component of quality assessment and benchmarking. The Pediatric Risk of Admission (PRISA) score is composed of descriptive, physiologic, and diagnostic variables that provide a probability of hospital admission as an index of severity. The score was developed and validated in a single tertiary pediatric hospital emergency department (ED) after exclusion of children with minor injuries and illnesses. We provide a multi-institutional recalibration and validation of the PRISA score and test its performance in 4 additional EDs, including patients with minor injuries and illnesses. Methods Masked, photocopied, randomly selected medical records of ED patients from 2000 were abstracted and were used to test the performance (discrimination and calibration) of the original PRISA score. This sample differed from the original PRISA sample by including 5 hospitals and including patients with minor injuries and minor illnesses. Independent variables included components of acute and chronic history, physiologic variables, and 3 ED therapies. The dependent variable was hospital admission. PRISA was then recalibrated as needed by using an 80% development sample and a 20% validation sample. Area under the curve and the Hosmer-Lemeshow goodness-of-fit test were used to measure, respectively, discrimination and calibration of the PRISA score after recalibration. We then applied the recalibrated PRISA score to secondary outcomes to test construct validity. We reasoned that a valid measure of ED severity should also be associated with the secondary outcomes of mandatory admissions (admissions using ≥1 inpatient resources) and ICU admissions. Results The recalibrated PRISA score performed well in all deciles of predicted probability of admission. The area under the curve was 0.81 and the calibration was good (Hosmer-Lemeshow 10.658; df=8; P=.222) for the development sample, and the area under the curve was 0.785 with excellent calibration (Hosmer-Lemeshow 8.341; df=9; P=.500) for the validation sample. The overall development sample had 423.9 admissions predicted and 423 observed; the validation sample had 112.1 predicted and 110 observed. Conclusion The PRISA score has been recalibrated and performs well in EDs of tertiary pediatric hospitals. Comparison with this benchmark may allow individual EDs to improve their performance and may provide insight into best practices.