Teamwork recognition of embodied agents with hidden Markov models

Recognizing and annotating the occurrence of team actions in observations of embodied agents has applications in surveillance or in training of military or sport teams. We describe the team actions through a spatio-temporal correlated pattern of movement, which can be modeled by a hidden Markov model. The hand-crafting of these models is a difficult task of knowledge engineering, even in application domains where explicit, natural language descriptions of the team actions are available. The main contribution of this paper is an approach through which the library of HMM representations can be acquired from a small number of hand annotated, representative samples of the specific movement patterns. A series of experiments, performed on a dataset describing a real-world terrestrial warfare exercise validates our method and shows good recognition accuracy even in the presence of noisy data. The speed of the recognition engine is sufficiently fast to allow real time annotation of incoming observations.