Efficient region-aware large graph construction towards scalable multi-label propagation

With fast growing number of images on photo-sharing websites such as Flickr and Picasa, it is in urgent need to develop scalable multi-label propagation algorithms for image indexing, management and retrieval. It has been well acknowledged that analysis in semantic region level may greatly improve image annotation performance compared to that in the holistic image level. However, region level approach increases the data scale to several orders of magnitude and proposes new challenges to most existing algorithms. In this work, we present a novel framework to effectively compute pairwise image similarity by accumulating the information of semantic image regions. Firstly, each image is encoded as Bag-of-Regions based on multiple image segmentations. Secondly, all image regions are separated into buckets with efficient locality-sensitive hashing (LSH) method, which guarantees high collision probabilities for similar regions. The k-nearest neighbors of each image and the corresponding similarities can be efficiently approximated with these indexed patches. Lastly, the sparse and region-aware image similarity matrix is fed into the multi-label extension of the entropic graph regularized semi-supervised learning algorithm [1]. In combination they naturally yield the capability of handling large-scale dataset. Extensive experiments on NUS-WIDE (260k images) and COREL-5k datasets validate the effectiveness and efficiency of our proposed framework for region-aware and scalable multi-label propagation.