Facial expression recognition based on mixture of basic expressions and intensities

Facial expression recognition can provide rich emotional information for human-robot interaction. This paper presents a facial expression recognition design that recognizes facial expressions as well as intensity and mixture ratio of six basic facial expressions. In this system, Active Appearance Model (AAM) and Lucas-Kanade image alignment algorithms are adopted to align the input facial images to obtain texture features. A novel method is proposed to recognize mixture ratio of basic facial expressions and the intensity of the expression. Three kinds of texture features are used in this method: 1. texture features of the whole face, which are used as inputs of facial expression intensity recognition; 2. texture features of the upside face, which are used as inputs of upper face action units recognition; 3. texture features of the downside face, which are used as the inputs of lower face action units recognition. Back propagation neural networks are used to obtain the recognition scores, which are then exploited to classify the facial expression results. Experimental results verified that the proposed method can effectively recognize mixture ratio of six basic expressions and the expression intensity.