Title :
Multiperson Visual Focus of Attention from Head Pose and Meeting Contextual Cues
Author :
Ba, Sileye O. ; Odobez, Jean-Marc
Author_Institution :
LabSTICC, Ecole Nat. des Telecommun. de Bretagne, Technopole Brest-Iroise, France
Abstract :
This paper introduces a novel contextual model for the recognition of people´s visual focus of attention (VFOA) in meetings from audio-visual perceptual cues. More specifically, instead of independently recognizing the VFOA of each meeting participant from his own head pose, we propose to jointly recognize the participants´ visual attention in order to introduce context-dependent interaction models that relate to group activity and the social dynamics of communication. Meeting contextual information is represented by the location of people, conversational events identifying floor holding patterns, and a presentation activity variable. By modeling the interactions between the different contexts and their combined and sometimes contradictory impact on the gazing behavior, our model allows us to handle VFOA recognition in difficult task-based meetings involving artifacts, presentations, and moving people. We validated our model through rigorous evaluation on a publicly available and challenging data set of 12 real meetings (5 hours of data). The results demonstrated that the integration of the presentation and conversation dynamical context using our model can lead to significant performance improvements.
Keywords :
audio signal processing; behavioural sciences computing; image motion analysis; pose estimation; VFOA; audio visual perceptual cues; context dependent interaction models; floor holding patterns; gazing behavior; head pose attention; meeting contextual cues; multiperson visual focus; social dynamics; visual focus of attention; Bayesian methods; Context modeling; Data models; Visualization; Visual focus of attention; contextual cues; conversational events; dynamic Bayesian network; head pose; meeting analysis.; multimodal; Algorithms; Artificial Intelligence; Attention; Bayes Theorem; Computer Simulation; Cues; Head Movements; Humans;
Journal_Title :
Pattern Analysis and Machine Intelligence, IEEE Transactions on
DOI :
10.1109/TPAMI.2010.69