Learning semantic visual dictionaries: A new method for local feature encoding

In this paper, we develop a new method to learn semantic visual dictionaries for local image feature encoding. Conventional methods usually learn dictionaries from random local image patches. Different from them, we manually select a number of object classes whose visual patterns can be seen at local image patch level in complex images (Figure 1), and learn dictionaries from their thumbnail images. The benefit is that these thumbnail images have class labels, so we can cluster semantically similar images together to generate meaningful cluster centers. Some other contributions of this paper include developing an adaptation method to adapt the learned dictionaries to target datasets, and developing efficient algorithms to encode local patches with our semantic visual dictionaries. Experimental results on three benchmark datasets demonstrate the effectiveness of the proposed methods.