A spectral method for context based disambiguation of image annotations

In this work we employ contextual information to improve the quality of image labellings provided by an existing automatic image annotation algorithm in a weakly supervised setting, where each training image is labelled but it is not known which part of the image its labels are referring to. We recast the problem into that of constructing a graph which encodes pairwise consistency of candidate annotations and observe that mutually consistent labels will form a compact cluster in this graph. We recover the clusters using a spectral theory based technique. The results are demonstrated on the Corel5k dataset. With improvements in the range of 25%-55% the performance in some cases approaches the state of the art despite using a very simple base algorithm.