Structurally enhanced latent semantic analysis for video object retrieval

The paper presents work aimed at reducing the semantic gap between low level video features and semantic video contents. The proposed method for finding associations between segmented frame region characteristics relies on the strength of latent semantic analysis (LSA). Previous work, using colour histograms and Gabor features, has rapidly shown the potential of this approach, but also uncovered some of its limitations. The use of structural information is necessary, yet rarely employed for such a task. The paper addresses two important issues: the first is to verify that using structural information does indeed improve information retrieval performances; the second concerns the manner in which this additional information is integrated within the framework. Two methods are proposed using the structural information contained in an object parts´ topological arrangement. The first adds structural constraints indirectly to the LSA during the preprocessing of the video, while the other includes the structure directly within the LSA. Finally, retrieval results demonstrate that, when the structure is added directly to the LSA, the performance gain of combining visual (low level) and structural information is convincing.