Multimodal surveillance based personal protection system

A novel, multimodal approach for automatic detection of abduction of a protected individual, employing dedicated personal protection device and a city monitoring system is proposed and overviewed. The solution is based on combining four modalities (signals coming from: Bluetooth, fixed and PTZ cameras, thermal camera, acoustic sensors). The Bluetooth signal is used continuously to monitor the protected person presence, and in case of abduction attempt it reports an alert accompanied with GPS coordinates. The video monitoring algorithm analyses streams from cameras closest to the event coordinates, examines the direction of objects´ movement and detects situations such as invaliding cars and road blocking. Thermal camera images are used for the detection of explosions and tracing cars in difficult lighting conditions. The audio monitoring subsystem uses acoustic sensors for the detection and localization of important sounds, such as shouts and gunshots. As a result, the combined modalities allow for the detection of important security threats, i.e. a person abduction in some studied case scenarios.