A multi-modality attributes representation scheme for Group Activity characterization and data fusion

Proper characterization of human Group Activity (GA) interactions can help to detect and prevent certain pertinent threats efficiently. In this paper, we present a model-based scheme for robust group activity characterization. The proposed approach takes advantage of synergy of multi-sensors data to track and identify key individual and group activity events based on fusion of imagery and acoustic sensors data. Each activity event is attributed by a set of tagged features. By matching and correlating attributes of events, the model attempts to associate sensory observations to a priori known ontology. The proposed model benefits from a fusion process that achieves perceptual grouping of activities by spatiotemporal correlation and association of fragmented perceptions extracted from attributed events. In this paper, we present the results of our experimental work and demonstrate the effective and robustness of the decision fusion technique in terms of properly classifying group activities and generating semantic messages describing dynamics of human group activities that, in turn, improves situational awareness.