Unsupervised Measures for Estimating the Effectiveness of Image Retrieval Systems

The main objective of Content-Based Image Retrieval (CBIR) systems is to retrieve a ranked list containing the most similar images of a collection given a query image, by taking into account their visual content. Although these systems represent a very promising approach, in many situations is very challenging to assure the quality of returned ranked lists. Supervised approaches rely on training data and information obtained from user interactions to identify and then improve low-quality results. However, these approaches require a lot of human efforts which can be infeasible for many systems. In this paper, we present two novel unsupervised measures for estimating the effectiveness of ranked lists in CBIR tasks. Given an estimation of the effectiveness of ranked lists, many CBIR systems can, for example, emulate the training process, but now without any user intervention. Improvements can also be achieved on several unsupervised approaches, such as re-ranking and rank aggregation methods, once the estimation measures can help to consider more relevant information by distinguishing effective from non-effective ranked lists. Both proposed measures are computed using a novel image representation of ranked lists and distances among images considering a given dataset. The objective is to exploit the visual patterns encoded in the image representations for estimating the effectiveness of ranked lists. Experiments involving shape, color, and texture descriptors demonstrate that the proposed approaches can provide accurate estimations of the quality in terms of effectiveness of ranked lists. The use of proposed measures are also evaluated in image retrieval tasks aiming at improving the effectiveness of rank aggregation approaches.