Augmented visual phrase in mobile product recognition

With the rapid advancement in mobile device technologies and connectivity, the use of mobile devices for visual object recognition is emerging as an application with great commercialization potentials. However, query images captured by mobile devices often suffer from various conditions such as illumination, scale, and viewpoint changes. To handle these, several detectors and descriptors have been proposed. However, recognition remains a challenge under strong photometric or geometric variation. In view of this, we propose a new Augmented Visual Phrase (AVP) framework that addresses this issue by using augmented features from transformed images. We propose a recognition framework based on the Bag of Phrase (BoP) structure which in turn is built on the Bag of Words (BoW) model. The proposed method can provide better performance by incorporating the spatial relationship of visual elements detected by keypoint detectors. To further eliminate spurious matches of visual phrases, Geometric Verification (GV) is applied to the top-ranked images. Experimental results show that the proposed AVP method outperforms the current BoP method by 9% in recognition rate.