Combining diagrammatic reasoning with qualitative spatial and temporal reasoning for motion event detection

Everyday spatio-temporal reasoning is driven through qualitative abstractions over mental maps or `diagrams´. Diagrammatic reasoning involves direct manipulation and inspection of diagrams as the primary means of inference. Diagrammatic representation offer computational advantage in problems where spatial relationships play a prominent role. In video, objects change spatial relationships over time. Therefore, combining diagrammatic reasoning with qualitative spatial and temporal reasoning holds promise. In this paper, we put forward a framework combining diagrammatic representation with qualitative spatial and temporal reasoning for motion event detection in video. Key frames with tracked objects for a given video are extracted. These frames in forward moving time are represented using specified `diagrams´. A set of perception function is defined exploiting results from diagrammatic reasoning and qualitative spatial and temporal reasoning for determining spatial relations between objects of interest. Inter diagrammatic reasoning operator is used to combine sequence of diagrams for extracting spatio-temporal changes. Diagram modification function is defined exploiting results from qualitative reasoning to extract directional information. Considering extracted relative position and relative direction of displacement as features, we use supervised machine learning techniques to recognize motion events in video. The approach is tested in videos with few people/groups meeting, walking together and splitting up/ fighting from the CAVIAR dataset.