Cross-modal attribute based facial image retrieval

Attribute-based facial image retrieval has wide range of applications, such as in law enforcement, online social networks, etc. The problem becomes more challenging if the images are from different modalities. For example, the input is a sketch or a composite image, and the task is to retrieve photo images which have the same facial attributes as the input data. In this work, we propose a learning-based approach, in which two transformations are learnt to transform the training images from the two modalities with associated attribute annotations such that images which have similar attributes move closer to each other, and images with very different attributes move farther from each other in the transformed space. Given a query image, it is first transformed to the learnt space in which the images with similar attributes are retrieved. The same framework works seamlessly if the images to be retrieved are of same or different modality as compared to the query data. The attributes of the query image are also automatically obtained as a byproduct of the algorithm. Extensive experimental evaluation on three datasets shows the effectiveness of the proposed approach.