Automatically Finding Images for Clinical Decision Support

Essential information is often conveyed in illustrations in biomedical publications. A clinician´s decision to access the full text when searching for evidence in support of clinical decision is frequently based solely on a short bibliographic reference. We seek to automatically augment these references with images from the article that may assist in finding evidence. The feasibility of automatically classifying images by usefulness (utility) in finding evidence was explored using supervised machine learning. We selected 2004 - - 2005 issues of the British Journal of Oral and Maxillofacial Surgery, manually annotating 743 images by utility and modality (radiological, photo, etc.) Image data, figure captions, and paragraphs surrounding figure discussions in text were used in classification. Automatic image classification achieved 84.3% accuracy using image captions for modality and 76.6% accuracy combining captions and image data for utility. Our results indicate that automatic augmentation of bibliographic references with relevant images is feasible.