Interactive voice response to assess residentsʹ laparoscopic skills: An instrument validation study

The study was undertaken to evaluate the psychometric properties of the Interactive Voice Response (IVR) instrument to assess laparoscopic surgical skills of residents. Study design Surgical competence of obstetric-gynecologic residents at the University of Ottawa (Canada) was evaluated after observed performance of skills. Three questions addressing general domains of surgical skill were rated per surgery using a 5-point Likert scale: overall, 993 surgical ratings were made by 29 residents and 13 faculty preceptors. Results With use of a generalizability study, a minimum of 12 preceptor ratings (G = 0.80) are needed to obtain reliable measures of residentsʹ surgical skills. Item analysis indicated that the three items on the instrument were highly correlated (Cronbach α = .92). Elements of construct validity were evident for diagnostic and simple laparoscopic procedures. Conclusion The IVR instrument can be a valid and reliable measure of residentsʹ surgical skills. The IVR ratings functioned as a global rating scale, capturing overall elements of surgical skills. The ratings on the IVR tool had construct validity with improved performance with increased surgical training and experience