Image retrieval with reciprocal and shared nearest neighbors

Content-based image retrieval systems typically rely on a similarity measure between image vector representations, such as in bag-of-words, to rank the database images in decreasing order of expected relevance to the query. However, the inherent asymmetry of k-nearest neighborhoods is not properly accounted for by traditional similarity measures, possibly leading to a loss of retrieval accuracy. This paper addresses this issue by proposing similarity measures that use neighborhood information to assess the relationship between images. First, we extend previous work on k-reciprocal nearest neighbors to produce new measures that improve over the original primary metric. Second, we propose measures defined on sets of shared nearest neighbors for reranking the shortlist. Both these methods are simple, yet they significantly improve the accuracy of image search engines on standard benchmark datasets.