Automatic bi-modal emotion recognition system based on fusion of facial expressions and emotion extraction from speech

Author

Datcu, Dragos ; Rothkrantz, Leon J M

Author_Institution

Fac. of Electr. Eng., Delft Univ. of Technol., Delft

fYear

2008

fDate

17-19 Sept. 2008

Firstpage

1

Lastpage

2

Abstract

Our software demo package consists of an implementation for an automatic human emotion recognition system. The system is bi-modal and is based on fusing of data regarding facial expressions and emotion that has been extracted from speech signal. We have integrated Viola & Jones face detector (OpenCV), active appearance model - AAM (AAM-API) for extracting the face shape and support vector machines (LibSVM) for the classification of emotion patterns. We have used optical flow algorithm for computing the features needed for the classification of facial expressions. Beside the integration of all processing components, the software system accommodates our implementation for the data fusion algorithm. Our C++ implementation has a working frame-rate of about 5 fps.

Keywords

emotion recognition; face recognition; feature extraction; image classification; image fusion; image sequences; speech recognition; support vector machines; AAM-API; C++ implementation; LibSVM; OpenCV; Viola & Jones face detector; active appearance model; automatic human bi-modal emotion recognition system; data fusion algorithm; emotion pattern classification; face shape extraction; facial expression classification; optical flow algorithm; software demo package; speech emotion extraction; support vector machine; Active appearance model; Active shape model; Data mining; Detectors; Emotion recognition; Face detection; Humans; Packaging; Software packages; Speech;

fLanguage

English

Publisher

ieee

Conference_Titel

Automatic Face & Gesture Recognition, 2008. FG '08. 8th IEEE International Conference on

Conference_Location

Amsterdam

Print_ISBN

978-1-4244-2153-4

Electronic_ISBN

978-1-4244-2154-1

Type

conf

DOI

10.1109/AFGR.2008.4813335

Filename

4813335