Fast Facial Fitting Based on Mixture Appearance Model with 3D Constraint

Fast facial points fitting plays an important role in applications such as Human-Computer Interaction, entertainment, surveillance, and is highly relevant to the techniques of facial expression analysis, face recognition, 3D face model generation, etc. Active Appearance Models (AAMs) are generative models commonly used to fit face. They are sensitive to illumination and expression changes because they use only raw intensity to build observation models. In this paper, a real time facial points fitting approach using mixture observation models is presented. Furthermore, the 3D modes are used to constrain the AAM so that it can only generate model instances that can also be generated with the 3D modes. Finally, we give a derivative process for fast energy minimization using the inverse compositional algorithm. A coarse-to-fine fitting strategy is used for realtime and robust facial points fitting. We apply this algorithm to facial expression cloning of 3D Avatar system. Experimental results demonstrate that fitting the AAM with mixture observation models and 3D constraint outperforms other classical algorithms.