Learning to combine multi-resolution spatially-weighted co-occurrence matrices for image representation

Bag-of-Words is widely used to describe images for image classification. However, this approach is limited because the spatial relation over visual words is not well exploited and also it is difficult to generate a single comprehensive vocabulary. In this paper, we propose novel effective schemes to handle these two issues. First, we propose a structure propagation technique to build more reasonable co-occurrence matrices of visual words to exploit the spatial information, which assigns a higher weight to the co-occurrence over two patches that lie in the same object part. Second, we build the multiple-histogram representation over hierarchical vocabularies to avoid the ambiguity of single vocabulary, and particularly present a learning approach to combine the multiple histograms to integrate both within-vocabulary and cross-vocabulary information. We evaluate our proposed method using the Princeton sports event dataset. Compared to the state-of-the-art results, our proposed approach has shown promising results.