Exploring Image Context for Semantic Understanding and Retrieval

Besides low-level visual features lots of researches focus on how to explore and utilize other kinds of related features for image semantic understanding and retrieval. Text is one of such related features. Some researches directly use semantic information from related texts to label image semantics, and ignore underlying low-level correlation. Differently, this paper explores low-level correlation between feature matrices of images and texts with kernel-based method; and then models semantic structure in the subspace based on manifold learning; we propose strategies to further refine manifold structure; also we discuss how to enable image retrieval with examples outside database. Our approach considers text as the context of images and uses content-based method to analyze statistical correlation between such context and image data. Users can submit a text or an image example to search similar images. Experiment and comparison results are encouraging and show that the performance of our approach is effective.