Comments on “Comparative Study With New Accuracy Metrics for Target Volume Contouring in PET Image Guided Radiation Therapy”

Phantom measurements with glass inserts in a hot background are still used frequently for calibration and performance assessment of volume delineation algorithms in positron emission tomography (PET). Taking as an example the recent paper “Comparative study with new accuracy metrics for target volume contouring in PET image guided radiation therapy” (Shepherd, 2012), we demonstrate that this is not a valid approach due to the discontinuity introduced in the background by the cold walls of the glass inserts. We, moreover, emphasize that in order to define a sensible ground truth for performance assessment of contouring algorithms in patient data it is necessary to average over a sizable number of experienced observers and lesions in order to compensate for the substantial inter-observer variability.