A Structural Texture Approach for Characterising Malignancy Associated Changes in Pap Smears Based on Mean-Shift and the Watershed Transform

This paper presents a novel structural approach to quantitatively characterising nuclear chromatin texture in light microscope images of Pap smears. The approach is based on segmenting the chromatin into blob-like primitives and characterising their properties and arrangement. The segmentation approach makes use of multiple focal planes. It comprises two basic steps: (i) mean-shift filtering in the feature space formed by concatenating pixel spatial coordinates and intensity values centred around the best all-in-focus plane, and (ii) hierarchical marker-based watershed segmentation. The paper also presents an empirical evaluation of the approach based on the classification of 43 routine clinical Pap smears. Two variants of the approach were compared to a reference approach (employing extended depth-of-field rather than mean-shift) in a feature selection/classification experiment, involving 138 segmentation-based features, for discriminating normal and abnormal slides. The results demonstrate improved performance over the reference approach. The results of a second feature selection/classification experiment, including additional classes of features from the literature, show that a combination of the proposed structural and conventional features yields a classification performance of 0.919 ± 0.015 (AUC ± Std. Dev.). Overall the results demonstrate the efficacy of the proposed structural approach and confirm that it is indeed possible to detect malignancy associated changes (MACs) in conventional Papanicolaou stain.