Graph-based retrieval of multi-modality medical images: A comparison of representations using simulated images

Content-based image retrieval (CBIR) is an image search technique that utilises visual features as search criteria; it has potential clinical applications in evidence-based diagnosis, physician training, and biomedical research. Graph-based CBIR techniques have high accuracy when retrieving images by the similarity of the spatial arrangement of their constituent objects but these techniques were initially designed for single-modality images and have limited retrieval capabilities when multi-modality images, such as combined positron emission tomography and computed tomography (PET-CT), are considered. In this paper, we present a graph-based CBIR approach for multimodality images that integrates modality-specific features on graph vertices and adapts a well-established graph similarity scheme to account for varying vertex feature sets. Furthermore, we propose a graph pruning method that removes redundant edges using the spatial proximity of image regions. We evaluated our work using two simulated data sets, consisting of 2D liver shapes and 3D whole-body lymphoma images. In our experiments we achieved a higher level of retrieval precision using our graph method when compared to conventional graph-based retrieval, demonstrating that our proposed method enabled new capabilities and improved multi-modality CBIR.