Optimized Window Arrangement for Spatial Pyramid Matching Scheme

Spatial Pyramid Matching (SPM) has become a standard in bag-of-words (BoW) image representation, regardless of the features used in the process. With most research focusing on other part of the BoW pipeline, the arrangement of spatial windows in SPM remains untouched except by few works. This paper takes into consideration the idea that not all spatial windows in SPM are needed, and proposes two systematic ways of learning the optimal spatial window arrangement, called greedy-OA and linear-OA. In both algortihms, the spatial windows of a SPM pyramid are examined in detail, and optimized to achieve the best performance with the lowest number of spatial windows possible. Using overlapping spatial windows from out previous work to enhance the discriminative power of each spatial windows, our experiments shows that the complete SPM-pyramid is indeed sub-optimal, as we are able to consistently achieve better performance (up to 4. 38%) with a reduced memory cost of 40%. Both algorithm are tested using 15 Scene, Caltech 101, and Caltech 256 datasets.