A method for evaluating the performance of content-based image retrieval systems

In recent years multimedia researchers have attempted to design content-based image retrieval systems. However, despite the development of these systems, the term "content" has still remained rather ill defined, and this has made the evaluation of such systems problematic. This paper proposes a method for the creation of a test set of images in which the similarity of each image pair is estimated by two independent methods - by the subjective evaluation of human observers, and by the use of "content words" as basis vectors that allow the multidimensional content of each image to be represented with a content vector. The latter method is shown to correlate with the subjective judgement of human observers, and thus provide both a more objective method for evaluating and expressing image content, and a possible path to automating the process in the future