Action unit reconstruction of occluded facial expression

Facial occlusion is a critical issue that may dramatically degrade the performance on facial expression-based emotion recognition. In this study, the Error Weighted Cross-Correlation Model (EWCCM) is employed to predict the facial Action Unit (AU) under partial facial occlusion from non-occluded facial regions for facial geometric feature reconstruction. In EWCCM, a Gaussian Mixture Model (GMM)-based Cross-Correlation Model (CCM) is first adopted to construct the statistical dependency among features from paired facial components such as eyebrows-cheeks of the non-occluded regions for AU prediction of the occluded region. A Bayesian classifier weighting scheme is then used to enhance the AU prediction accuracy considering the contributions of the GMM-based CCMs. Based on the predicted AU, a regression fusion scheme is proposed to reconstruct the occluded facial geometric features. Experimental results show that the proposed approach yielded satisfactory results on the NCKU-FEPO database for facial AU reconstruction.