Augmenting descriptors for fine-grained visual categorization using polynomial embedding

Fine-grained visual categorization (FGVC), which is a relatively new research area, distinguishes conceptually and visually similar categories such as plant and animal species. While FGVC is expected to lead to many task-specific practical applications, it is known as an extremely difficult problem because interclass variations are often quite subtle. We believe that the key to FGVC is improving local descriptors to enhance discriminative power at the local patch-level. While the pooling strategy of descriptors has been intensively improved for bag-of-visual-words (BoVW) based image representations, the descriptors themselves are often untouched. In this paper, we propose a descriptor augmentation method that utilizes polynomial embedding and supervised dimensionality reduction. Since our method provides moderate-sized compressed descriptors, it can be naturally integrated with off-the-shelf BoVW techniques. In experiments, we show that our method achieves state-of-the-art performance on standard FGVC datasets, Caltech-Birds, and Oxford-Flowers.