Improving Performance of Medical Images Retrieval by Combining Textual and Visual Information

This paper studies the combination of textual and visual information in a database of medical records in order to improve the performance of the multi-modal information retrieval system. The proposed model consists of two subsystems: a content-based information retrieval subsystem that performs the image retrieval and a textual information retrieval subsystem that performs the textual retrieval. The images and text are independently retrieved and then the partial resulting lists are mixed. A study of different weighting schemes has been accomplished and analyzed. The results obtained show that the proper integration of textual information improves conventional multi-modal systems.