A Real Time Spatial/Temporal/Motion Integrated Surveillance System in Compressed Domain

We designed a real time moving object extraction and behavior analysis system to detect if any abnormal behavior like intrusion, halt, and wall climbing happened. Much work in the area of motion based video object segmentation is being done in the pixel domain which exploits the visual attributes and motion information. Given most existing images and videos stored in the compressed form, the specific manipulation algorithms can be applied to the compressed streams without full decoding of the compressed images/videos. In this paper, we could produce motion masks from I and P frames in MPEG video. Then, motion vectors decoded directly from P frames could be utilized to do object extraction. By clustering motion vectors conforming to the same motion, we could extract the moving regions. If moving regions are connected closely and of the same direction, they are claimed to be of the same moving object. Following the motion vectors in consecutive P frames, we could track the moving objects. A confidence is defined from these matched blocks. Finally, a couple of criterions are defined for object behavior analysis whether they intrude a monitored area, halt on road, or climb wall. Experimental results are given to demonstrate the feasibility of our system.