Weighting scheme for image retrieval based on bag-of-visual-words

Inspired by the success of bag-of-words in text retrieval, bag-of-visual-words and its variants are widely used in content-based image retrieval to describe visual content. Various weighting schemes have also been proposed to integrate different yet complementary visual-words. However, most of these weighting schemes tend to use fixed weight for every visual-word extracted from the query image, which may lose the discriminative information. This study presents a novel combining method which captures query-specific weights for visual-words in query image. The method mainly contains two stages. Firstly, in offline weight learning, the authors introduce a linear classifier to build a query-category mapping table, and max-margin learning to build category-weight mapping table. Query-category mapping table is used to map the query image to the most likely image class, and category-weight mapping table is used to map image class to the weights of visual-words. Secondly, in online weight mapping, the weights of visual-words are determined efficiently by looking into the pre-learned mapping tables. Experimental results on WANG database and Caltech 101 demonstrate that the proposed weighting scheme can effectively weight visual-words of query image according to their discriminative information. In addition, comparative experiments demonstrate the proposed weighting scheme can obtain higher retrieval performance than other weighting schemes.