Detecting Semantic Group Activities Using Relational Clustering

Existing approaches to detect modeled activities in video often require the precise specification of the number of actors or roles, or spatial constraints, or other limitations that create difficulties for generic detection of group activities. We develop an approach to detect group behaviors in video, where an arbitrary number of participants are involved. We address scene conditions with non-participating objects, an arbitrary number of instances of the behaviors of interest, and arbitrary locations for those instances. Our approach uses semantic spatio-temporal predicates to define activities, and relational clustering to identify groups of objects for which the relational predicates are mutually true over time. The algorithm handles conditions where object segmentation and tracking are highly unreliable, such as busy scenes with occluders. Results are shown for the group activities of crowd formation and dispersal on low-resolution, far-field video surveillance data.