A Unified Regularization Framework for Virtual Frontal Face Image Synthesis

Taking advantage of the statistical learning-based point of view, several approaches of frontal face image synthesis have received remarkable achievement. However, the existing methods mainly utilize either ordinary least squares (OLS) or fixed l₁-norm penalized sparse regression to estimate the solution. For the former, the solution is unstable when the linear equations system is ill-conditioned. For the latter, sparsity is only considered, while the significance of local similarity between input image and each training sample is ignored. Thus the synthesized result fails to faithfully approximate the ground truth. Moreover, these traditional methods cannot ensure the consistency between corresponding patches in frontal and profile faces. To address these problems, we present a unified regularization framework (URF) by imposing two regularization terms onto the solution. Firstly, we introduce an l2-norm constraint and impose a diagonal weights matrix onto it, in which each diagonal entry is defined by the spatial distance between input image patch and individual patch in training set. Secondly, to mitigate the aforementioned inconsistency problem, we present a neighborhood consistency regularization term, motivated by manifold learning. Finally, we generalize our framework to the lq-norm penalized case. By adjusting the shrinkage parameter q, the framework gets more flexibility to choose a reasonable sparse domain. Extensive experiments on CMU Multi-PIE database and CAS-PEAL-R1 database verify the efficacy of our method.