Human emotional state recognition using real 3D visual features from Gabor library

Emotional state recognition is an important component for efficient human–computer interaction. Most existing works address this problem using 2D features, which are usually sensitive to head pose, clutter, and variations in lighting conditions. The existing 3D based methods only consider geometric information for visual feature extraction. In this paper, we introduce a method based on real 3D visual features for human emotion recognition. The 3D geometric information plus color/density information of the facial expressions are extracted by 3D Gabor library to construct visual feature vectors. The filter’s scale, orientation and shape of the library are specified according to the appearance patterns of the 3D facial expressions. An improved kernel canonical correlation analysis (IKCCA) algorithm is adopted for final decision. From training samples, the semantic ratings that describe the different facial expressions are computed by IKCCA to generate a seven dimensional semantic expression vector. It is then applied for learning the correlation with different testing samples. According to this correlation, we estimate the associated expression vector and perform expression classification. From experimental results, our proposed method demonstrates impressive performance.