Anomaly detection with spatio-temporal context using depth images

A novel statistical framework for modeling the intrinsic structure of crowded scenes and detecting abnormal activities is presented. The proposed framework essentially turns the complex anomaly detection process into two parts: motion pattern representation and spatio-temporal context modeling. We propose a new 4D spatio-temporal hypervolume representation by integrating the depth constraints to enrich motion information. When detecting abnormal behaviors from crowded scenes, we divide the hypervolume into local blocks and construct environmental contexts by coupling their spatio-temporal correlations together with the co-occurrence probabilities. As a result, statistical deviations can be detected as abnormal events. Experiments on a new depth image dataset composed of four crowded scene categories show that our spatiotemporal framework offers promising results in real-life crowded scenes with complex activities.