Real-time understanding of 3D video on an embedded system

A method is proposed for extracting semantic knowledge from 3D video in real-time on an embedded system. The method consists of foreground segmentation, object segmentation, temporal tracking and classification stages. It assigns relevant objects to different classes (e.g., human, vehicle) and determines their spatial location. These features are used in numerous applications including intrusion detection, people or vehicles counting, incident detection and activity monitoring. On a DSP-based embedded system, people recognition and object localization have been performed at over 10 frames per second. In the process, a time-of-flight range imaging camera has been used for video input. The experiments have also shown that the method is capable of coping with partial occlusion situations when different objects are located on distinct depth planes.