Title :
Spatiotemporal semantic video segmentation
Author :
Galmar, E. ; Athanasiadis, Th ; Huet, B. ; Avrithis, Y.
Author_Institution :
Dept. Multimedia, Eurecom, Sophia Antipolis
Abstract :
In this paper, we propose a framework to extend semantic labeling of images to video shot sequences and achieve efficient and semantic-aware spatiotemporal video segmentation. This task faces two major challenges, namely the temporal variations within a video sequence which affect image segmentation and labeling, and the computational cost of region labeling. Guided by these limitations, we design a method where spatiotemporal segmentation and object labeling are coupled to achieve semantic annotation of video shots. An internal graph structure that describes both visual and semantic properties of image and video regions is adopted. The process of spatiotemporal semantic segmentation is subdivided in two stages: Firstly, the video shot is split into small block of frames. Spatiotemporal regions (volumes) are extracted and labeled individually within each block. Then, we iteratively merge consecutive blocks by a matching procedure which considers both semantic and visual properties. Results on real video sequences show the potential of our approach.
Keywords :
feature extraction; graph theory; image matching; image segmentation; image sequences; spatiotemporal phenomena; video signal processing; feature extraction; image segmentation; image sequences; internal graph structure; matching procedure; spatiotemporal semantic video segmentation; video shot; Computational efficiency; Data mining; Image segmentation; Knowledge representation; Labeling; Layout; Multimedia databases; Spatiotemporal phenomena; Video sequences; Visual databases;
Conference_Titel :
Multimedia Signal Processing, 2008 IEEE 10th Workshop on
Conference_Location :
Cairns, Qld
Print_ISBN :
978-1-4244-2294-4
Electronic_ISBN :
978-1-4244-2295-1
DOI :
10.1109/MMSP.2008.4665143