Learning of Perceptual Similarity From Expert Readers for Mammogram Retrieval

Content-based image retrieval relies critically on the use of a computerized measure of the similarity (i.e., relevance) of a query image to other images in a database. In this work, we explore a superivised learning approach for retrieval of mammogram images, of which the goal is to serve as a diagnostic aid for breast cancer. We propose that the most meaningful measure is one that is designed specifically to match that perceived by the radiologists in their interpretation of mammogram lesions. In our approach, we model the notion of similarity as an unknown function of the image features characterizing the lesions, and use modern machine-learning algorithms to learn this function from similarity scores collected from radiologists in reader studies. This approach is evaluated using data collected from an observer study with a set of clinical mammograms. Our results demonstrate that the proposed machine learning approach can be used to model the notion of similarity as judged by expert readers in their interpretation of mammogram images and that it can outperform alternative similarity measures derived from unsupervised learning.