Detecting rare events in video using semantic primitives with HMM

We present a new approach for recognizing rare events in aerial video. We use the framework of hidden Markov models (HMMs) to represent the spatio-temporal relations between objects and uncertainty in observations, where the data observables are semantic spatial primitives encoded based on prior knowledge about the events of interest. Events are observed as a sequence of binarized distance relations among the objects participating in the event. This avoids directly modeling the temporal trajectories of continuous observables, which is difficult when training data is scarce. The approach enables better generalization to other scenes for which little or no training data may be available. We demonstrate the effectiveness of our approach using real aerial video and simulated data.