Facial expression invariant head pose normalization using Gaussian Process Regression

We present a regression-based scheme for facial-expression-invariant head pose normalization. We address the problem by mapping the locations of 2D facial points (e.g. mouth corners) from non-frontal poses to the frontal pose. This is done in two steps. First, we propose a head pose estimator that maps the input 2D facial point locations into a head-pose space defined by a low dimensional manifold attained by means of multi-class LDA. Then, to learn the mappings between a discrete set of non-frontal head poses and the frontal pose, we propose using a Gaussian Process Regression (GPR) model for each pair of target poses (i.e. a non-frontal and the frontal pose). During testing, the head pose estimator is used to activate the most relevant GPR model which is later applied to project the locations of 2D facial landmarks from an arbitrary pose (that does not have to be one of the training poses) to the frontal pose. In our experiments we show that the proposed scheme (i) performs accurately for continuous head pose in the range from 0° to 45° pan rotation and from 0° to 30° tilt rotation despite the fact that the training was conducted only on a set of discrete poses, (ii) handles successfully both expressive and expressionless faces (even in cases when some of the expression categories were missing in certain poses during the training), and (iii) outperforms both 3D Point Distribution Model (3D-PDM) and Linear Regression (LR) model that are used as baseline methods for pose normalization. The proposed method is experimentally evaluated on data from the BU-3DFE facial expression database.