A learning-based approach for annotating large on-line image collection

Several recent works attempt to automatically annotate image collection by exploiting the links between visual information provided by segmented image features and semantic concepts provided by associated text. The main limitation of such approaches, however, is that semantically meaningful segmentation is in general unavailable. This paper proposes a novel statistical learning-based approach to overcome this problem. We employ two different segmentation methods to segment the image into two sets of regions and learn the association between each set of regions with text concepts. Given a new image, the idea is to first employ a greedy strategy to annotate the image with concepts derived from different sets of overlapping and possibly conflicting regions. We then incorporate a decision model to disambiguate the concepts learned using the visual features of the overlapping regions. Experiments on a mid-sized image collection demonstrate that the use of our disambiguation approach could improve the performance of the system by about 12-16% on average in terms of F₁ measures as compared to system that uses only one segmentation method.