Simultaneous validation of image segmentation and assessment of expert quality [tumor MRI application]

Characterizing the performance of image segmentation approaches has been a persistent challenge. Interactive drawing of the desired segmentation by domain experts has often been the only acceptable approach, and yet suffers from intra-expert and inter-expert variability. Algorithms have been sought in order to remove the variability introduced by experts. The accuracy of segmentations of medical images has been difficult to quantify in the absence of a "ground truth" segmentation for clinical data. Although physical or digital phantoms can help, they have so far been unable to reproduce the full range of imaging and anatomical characteristics observed in clinical data. An attractive alternative is comparison to a collection of segmentations by experts, but the most appropriate way to compare segmentations has been unclear. We present here a method for computing a maximum likelihood estimate of the "ground truth" segmentation from a group of expert segmentations, and a simultaneous measure of the quality of each expert. This approach readily enables the assessment of an automated image segmentation algorithm, and direct comparison of expert and algorithm performance.