A hierarchical grid feature representation framework for automatic image annotation

We propose a hierarchical-grid (HG) feature analysis framework for representing images in automatic image annotation (AIA). We explore the properties of codebooks constructed with different-sized grids in image sub-blocks, and co-occurrence relationship between VQ codewords constructed from different grid systems. The proposed HG approach is evaluated on the TRECVID 2005 data set using classifiers obtained with maximal figure-of-merit discriminative training. With multi-level and cross-level grid systems incorporating bigram information within and between higher and lower grid levels, we show that the AIA performance can be significantly improved. For 20 selected concepts from the 39-concept LSCOM-Lite annotation set, we achieve a best F₁ in almost all the concepts. The overall performance improvement with the combined multi-level and cross-level grid systems over the best single-size grid system in micro F₁ is about 12.1%.