Evaluation of Multi-Sensor Surveillance Event Detectors

This paper describes the challenge for real-time event detection for surveillance applications (CREDS). CREDS consists in an evaluation protocol for a specific set of event detection problems, a multi-camera dataset and the corresponding ground-truth annotation, and an automatic evaluation metric. The key features of this challenge are the adoption of a top-down approach (i.e., the requirements are defined by an end-user) and an easy definition of the ground-truth annotation, which can be generated quickly. We analyze the elements of the challenge, its implementation and the results from the evaluation protocol. Moreover, we propose a unifying summary of the video content analysis algorithms that participated in the challenge. Finally, to facilitate progress in the use of iterative algorithm self-testing and inter-algorithm comparisons, we suggest how current evaluation efforts can be extended towards the definition of a globally accepted evaluation protocol and testing database