Title :
Evaluation and Discussion of Multi-modal Emotion Recognition
Author :
Rabie, Ahmad ; Wrede, Britta ; Vogt, Thurid ; Hanheide, Marc
Author_Institution :
Appl. Inf. Group, Bielefeld Univ., Bielefeld, Germany
Abstract :
Recognition of emotions from multimodal cues is of basic interest for the design of many adaptive interfaces in human-machine and human-robot interaction. It provides a means to incorporate non-verbal feedback in the interactional course. Humans express their emotional state rather unconsciously exploiting their different natural communication modalities. In this paper, we present a first study on multimodal recognition of emotions from auditive and visual cues for interaction interfaces. We recognize seven classes of basic emotions by means of visual analysis of talking faces. In parallel, the audio signal is analyzed on the basis of the intonation of the verbal articulation. We compare the performance of state of the art recognition systems on the DaFEx database for both complement modalities and discuss these results with regard to the theoretical background and possible fusion schemes in real-world multimodal interfaces.
Keywords :
emotion recognition; human computer interaction; human-robot interaction; DaFEx database; audio signal; auditive cue; human-machine interaction; human-robot interaction; interaction interfaces; interactional course; multimodal emotion recognition; multimodal interfaces; natural communication modality; nonverbal feedback; verbal articulation; visual analysis; visual cue; Application software; Computer interfaces; Computer science; Emotion recognition; Face recognition; Humans; Informatics; Man machine systems; Psychology; Visual databases; facial expressions; multimodal integration; visual and speech based emotion recognition;
Conference_Titel :
Computer and Electrical Engineering, 2009. ICCEE '09. Second International Conference on
Conference_Location :
Dubai
Print_ISBN :
978-1-4244-5365-8
Electronic_ISBN :
978-0-7695-3925-6
DOI :
10.1109/ICCEE.2009.192