Pose-Invariant Facial Expression Recognition Based on 3D Face Reconstruction and Synthesis from a Single 2D Image

In this paper, a novel method is proposed for person-independent pose-invariant facial expression recognition based on 3D face reconstruction from only 2D frontal images in a training set. A 3D Facial Expression Generic Elastic Model (3D FE-GEM) is proposed to reconstruct an expression-invariant 3D model of each human face in the present database using only a single 2D frontal image with/without facial expressions. Then, for each 7-class of facial expressions in the database, a Feature Library Matrix (FLM) is created from yaw face poses by the rotating the 3D reconstructed models and extracting features in rotated face. Each FLM is subsequently rendered based on yaw angles of face poses. Before matching to the FLM, an initial estimate of yaw angles of face poses is obtained in the test face image using an automatic head pose estimation approach. Then, an array of the FLM is selected based on the estimated yaw angles for each class of facial expressions. Finally, the selected arrays from FLMs are compared with target image features by Support Vector Machine (SVM) classification. Favorable outcomes were acquired to handle pose in facial expression recognition on the available image based on the proposed method compared to several state-of-the-arts in pose-invariant facial expression recognition.