DocumentCode :
1756679
Title :
Face Expression Recognition by Cross Modal Data Association
Author :
Tawari, Ashish ; Trivedi, Mohan Manubhai
Author_Institution :
Comput. Vision & Robot. Res. Lab., Univ. of California, San Diego, La Jolla, CA, USA
Volume :
15
Issue :
7
fYear :
2013
fDate :
Nov. 2013
Firstpage :
1543
Lastpage :
1552
Abstract :
We present a novel facial expression recognition framework using audio-visual information analysis. We propose to model the cross-modality data correlation while allowing them to be treated as asynchronous streams. We also show that our framework can improve the recognition performance while significantly reducing the computational cost by avoiding redundant or insignificant frame processing by incorporating auditory information. In particular, we design a single good image representation of image sequence by weighted sums of registered face images where the weights are derived using auditory features. We use a still image based technique for the expression recognition task. Our framework, however, can be generalized to work with dynamic features as well. We performed experiments using eNTERFACE´05 audio-visual emotional database containing six archetypal emotion classes: Happy, Sad, Surprise, Fear, Anger and Disgust. We present one-to-one binary classification as well as multi-class classification performances evaluated using both subject dependent and independent strategies. Furthermore, we compare multi-class classification accuracies with those of previously published literature which use the same database. Our analyses show promising results.
Keywords :
audio-visual systems; emotion recognition; face recognition; image classification; image representation; image sequences; archetypal emotion classes; asynchronous streams; audio-visual information analysis; auditory features; auditory information; cross modal data association; cross-modality data correlation; eNTERFACE´05 audio-visual emotional database; facial expression recognition framework; image representation; image sequence; multiclass classification accuracies; multiclass classification performances; one-to-one binary classification; Facial expression recognition; affect analysis; affective computing; audio-visual expression recognition; emotion recognition; key frames selection; multi-modal expression recognition;
fLanguage :
English
Journal_Title :
Multimedia, IEEE Transactions on
Publisher :
ieee
ISSN :
1520-9210
Type :
jour
DOI :
10.1109/TMM.2013.2266635
Filename :
6525327
Link To Document :
بازگشت