Robust modified Active Shape Model for automatic facial landmark annotation of frontal faces

In this paper we present an improved method for locating facial landmarks in images containing frontal faces using a modified active shape model. Our main contributions include the use of an optimal number of facial landmark points, better profiling methods during the fitting stage and the development of a more suitable optimization metric to determine the best location of the landmarks compared to the simplistic minimum Mahalanobis distance criteria used to date. We build a subspace to model variations of appearance around each facial landmark and use this subspace to enhance the accuracy of the fitting process around each landmark. This enhancement provides a significant improvement in fitting and simultaneously determines which points were poorly fitted using reconstruction error, thus allowing for automatic correction or interpolation of any poorly fitted points. Our implementation, with the above mentioned improvements, leads to extremely accurate results even when dealing with faces with expressions, slight pose variations and in-plane rotations. Experiments conducted on test sets drawn from three databases (NIST Multiple Biometric Grand Challenge-2008 (MBGC-2008), CMU Multi-PIE and the Japanese Female Facial Expression (JAFFE) database) show that our proposed approach leads to far better performance compared to the classical active shape model of Cootes et al. and other traditional methods and provides a robust automatic facial landmark annotation which is the first critical step in face registration, pose correction and face recognition.