Automatic social interaction analysis with audio and visual nonverbal cues

In this paper we present a review of human social interaction analysis based on audio and visual nonverbal cues. Furthermore, as an example study, we present our study on automatic dominance estimation in small group conversations. We extracted low level audio and visual features, defined in parallel to the nonverbal cues displayed by dominant people, as stated in social psychology literature. We show that, using simple features and simple classifiers, we are able to achieve performances around 85-90% in estimating the most/least dominant person. We also show that audio features alone give high accuracies whereas visual features are necessary for more accurate results for the estimation of dominance.