Quality assurance kappa: field testing of a prototypical software program for evaluation of intraoperative transesophageal echocardiography skills

The authors have developed software (Q/A Kappa), involving a 400-line source code, capable of directly reporting Kappa correlation coefficient values for systematic assessment of the validity of intraoperative anesthesiology based echocardiographic interpretations, using external reviewer interpretations as the "gold standard". This study presents results of field testing of this prototypical software. Data, derived from consecutive cardiac surgery patients, consisted of standardized two-dimensional echocardiographic and color flow Doppler imaging sequences. Intraoperative and off-line TEE interpretations were compared using the Q/A Kappa program. These data (480 interpretations) were also manually integrated into the equation required for calculation of values of the variable kappa correlation coefficient. The relationship between Q/A Kappa-derived values and manually-calculated values was significant (P<0.001, r=1.0). This result indicates successful seamless functioning of this software program from data entry, through segmentation into tables, to valid statistical analysis calculations.