Ontology-coupled active contours for dynamic video scene understanding

In this paper, we present an innovative approach coupling active contours with an ontological representation of knowledge, in order to understand scenes acquired by a moving camera and containing multiple non-rigid objects evolving over space and time. The developed active contours enable both segmentation and tracking of multiple targets in each captured scene over a video sequence with unknown camera calibration. Hence, this active contour technique provides information on the objects of interest as well as on parts of them (e.g. shape and position), and contains simultaneously low-level characteristics such as intensity or color features. The ontology we propose consists of concepts whose hierarchical levels map the granularity of the studied scene and of a set of inter- and intra-object spatial and temporal relations defined for this framework, object and sub-object characteristics e.g. shape, and visual concepts like color. The system obtained by coupling this ontology with active contours can study dynamic scenes at different levels of granularity, both numerically and semantically characterize each scene and its components i.e. objects of interest, and reason about spatiotemporal relations between them or parts of them. This resulting knowledge-based vision system was demonstrated on real-world video sequences containing multiple mobile highly-deformable objects.