Automatic Face Modeling and Synthesis Based on Image Pairs

Unlike traditional 3D model based or image based animation methods, in this paper a novel approach is presented to generate both facial actions and head rotations for photo-realistic facial animation based on one frontal and one half-profile facial image taken with an uncalibrated camera. We represent faces with 2D wire-frame models and use MPEG4 FAPs to encode basic facial actions. Hierarchical Direct Appearance Model is employed for facial feature localization. 3D deformable model is applied for pose estimation. By affine projection 3D deformable model and facial actions are mapped to 2D facial models and actions at various head poses. Coarse 2D models are refined with extracted facial features by RBF interpolation. Pose-variable facial animation is generated by synthesizing facial actions on 2D models and morphing facial textures between frontal and halfprofile views. Experimental results demonstrate the effectiveness of our approach.