A semantically significant visual representation for social image retrieval

Having effective methods to access the desired images is essential nowadays with the availability of a huge amount of digital images. We propose a higher-level visual representation that enhances the traditional part-based Bag of Visual Words (BOW) representation in two aspects. Firstly, we introduce a new multilayer semantic significance analysis (MSSA) model to select Semantically Significant Visual Words (SSVWs) from the classical visual words in order to overcome the noisiness of the feature quantization process. Secondly, we strengthen the discrimination power of SSVWs by constructing Semantically Significant Visual Phrases (SSVPs) from frequently co-occurring SSVWs in the same local context that are semantically coherent. Finally, the large-scale extensive experimental results show that the proposed higher-level visual representation outperforms the traditional part-based image representation in social image retrieval.