On the Scalability and Adaptability for Multimodal Retrieval and Annotation

This paper presents a highly scalable and adaptable co-learning framework on multimodal image retrieval and image annotation. The co-learning framework is based on the multiple instance learning theory. While this framework is a general framework that may be used in any specific domains, to evaluate this framework, we apply it to the Berkeley Drosophila ISH embryo image database for the evaluations of the retrieval and annotation performance. In addition, we also apply this framework to across-stage inferencing for the embryo images for knowledge discovery. We have compared the performance of the framework for retrieval, annotation, and inferencing on the Berkeley Drosophila ISH database with a state-of-the-art multimodal image retrieval and annotation method to demonstrate the effectiveness and the promise of the framework.