Extending MPEG-7 description scheme of moving regions by the semantic visual-spatio-temporal relationships

The recent proliferation of multimedia contents led to the need to more effective and efficient content representation techniques to speed up their retrieval. MPEG-7 is a new standard that aims at describing the low-level (syntactic) and high-level (semantic) multimedia content. In MPEG-7, the relationships between moving regions (i.e. objects) in videos are represented by the directional, spatial and temporal relationships. In this paper, we propose to extend the description scheme (DS) of moving objects to include rich sets of visual-spatio-temporal (VST) relationships that support semantical descriptions of the relationships between objects. Also, we propose an XML based DSs of the VST relationships and then present the bit format representations for the VST relationships. The VST relationships are more intuitive to users, thus simplifying the formulation of user queries.