Template-Based Multiple Codebooks Generation for Fine-Grained Shopping Classification and Retrieval

Visual codebook based quantization of robust appearance descriptors extracted from local image patches is an effective means of capturing image statistics for object classification. A codebook is usually constructed by using a cluster method such as k-means at object level or image level. The codebook is global. For fine-grained categorization and recognition problems, however, the global object-level codebook can not reveal subtle differences among sub-categorization. Because different parts for an object have different appearances and statistical natures, the key to identify the fine-grained differences lies in finding the right alignment of image regions that contain the same object part. We adopt a template model that captures common shape patterns of object parts. With the template model, the image parts are aligned. Then, we generate multiple local codebooks, each for one template. Each local codebook is a "terminological and spatially local dictionary" for the corresponding template and can present more precisely context of the part. In addition, the sizes of different local codebooks may be different according to the context of the part. Experiment results on two datasets show that multiple local codebooks improve the existing methods. In addition, smaller size of codebook reduces computational time for codebook generation.