Saliency-aware semantic image coding for mobile visual search

Traditional image coding has mainly been concerned about its rate-distortion performance in the sense of what rate can be achieved for a given distortion that is usually measured by either peak signal to noise ratio (PSNR) or subjective quality. However, in many visual analysis scenarios, especially for mobile visual search (MVS), the rate-accuracy performance is crucial. In this paper, we propose a saliency-aware semantic image coding scheme to preserve the discriminative semantic information for visual search and assess the coding performance in the context of MVS, instead of traditional PSNR or subjective quality. We utilize saliency detection to obtain the region of interest (ROI) involving significant semantic information and encode the query image based on the framework of JPEG 2000 with ROI coding. The retrieval accuracy is evaluated on UKBench dataset. Experiment results show that, compared with regular JPEG 2000 without saliency detection, on average 12% and 17% bitrate can be saved by our scheme to achieve similar retrieval accuracy, respectively, for two bag-of-visual words based search methods.