Toward contextual forensic retrieval for visual surveillance: Challenges and an architectural approach

In traditional visual surveillance systems, retrieval has been relying on indexing events and features extracted by visual analytic algorithms that were developed for well-defined, specific domains. However, due to the increasing need for intelligent forensic retrieval with contextual semantics, this approach is reaching its limits, because it is almost impossible to predict and model all situations at development time. Consequently, a more flexible and intelligent retrieval approach is required. The goal of this paper is to explore the scope of requirements and architectural options to solve this problem. We consider several query scenarios inspired by real events that would benefit from intelligent support. We derive challenges and requirements by reviewing state-of-the-art retrieval approaches in terms of the selected queries. Based on the derived requirements, we present and discuss our architecture and its prototypical implementation.