Review of medical image retrieval systems and future directions

This paper presents a review of online systems for content-based medical image retrieval (CBIR). The objective of this review is to evaluate the capabilities and gaps in these systems and to determine ways of improving relevance of multi-modal (text and image) information retrieval in the iMedline system, being developed at the National Library of Medicine (NLM). Seven medical information retrieval systems: Figuresearch, BioText, GoldMiner, Yale Image Finder, Yottalook, Image Retrieval for Medical Applications (IRMA), and iMedline have been evaluated here using the system of gaps defined in [1]. Not all of these systems take advantage of the visual information contained in biomedical literature as figures and illustrations. However, all attempt to extract metadata about the image from the full-text of the articles and retrieve figures/images in response to a query. iMedline aims to advance the state-of-the-art in multimodal information retrieval by unifying image and text features in computing relevance. We discuss the shortcomings of these current systems and discuss future directions and next steps in iMedline toward context-based medical image retrieval.