Mining Cross-Modal Association Rules for Web Image Retrieval

To alleviate the known semantic gap, it is necessary to integrate the two-modal parts of Web images, i.e. the low-level visual features and high-level semantic concepts (which are usually represented by keywords), for Web image retrieval. In this paper, we associate the keyword and visual features of Web images from a different prospective and a new approach based on the cross-modal association rules is proposed to automatically integrate the keyword and visual features for Web image retrieval. A customized mining process is developed for the special association rule that crosses the two modals of Web images. The cross-modal association rule effectively associates the keyword and visual feature clusters, and seamlessly integrates the two modals of Web images in retrieval process. The proposed approach is utilized successfully in a Web image retrieval system named VAST (VisuAl & SemanTic image search).