Spatio-temporal modeling of facial expressions using Gabor-wavelets and hierarchical hidden Markov models

As one of the key techniques for futuristic man-machine interface, facial expression analysis has received much attention in recent years. This paper proposes a hierarchical approach to facial expression recognition in image sequences by exploiting both spatial and temporal characteristics within the framework of hierarchical hidden Markov models (HHMMs). Human faces are automatically detected in the maximum likelihood sense. Gabor-wavelet based features are extracted from image sequences to capture the subtle changes of facial expressions. Four prototype emotions; i.e. happiness, anger, fear and sadness, are investigated using the Cohn-Kanade database and an average of 90.98% person-independent recognition rate is achieved. We also demonstrate that HHMMs outperform HMMs for modeling image sequences with multilevel statistical structure.