Face recognition improvement by converting expression faces to neutral faces

A face recognition database generally consists of expressionless, frontal face images often referred to as neutral faces. However, we often obtain a facial image from a non-frontal view that may even contain expressions such as anger, joy, surprise, smile, sorrow, and etc. Faces with expressions often cause the underlying face recognition algorithm to fail. In this paper, we present an approach to improve face recognition by warping an image with facial expression to create a neutral, expression-invariant face. We use a modified version of the thin plate splines warping to remove the expression from a probe image with expressions to improve the correctness in face recognition using a gallery of neutral faces. We evaluate our proposed method using 2 well-known facial expression databases; namely, the AR-Face and MUG-FED databases. The experimental results for both databases show that our proposed method significantly improves the accuracy of face recognition under expression variations for the 3 commonly used approaches to face recognition including principal component analysis (PCA), linear discriminant analysis (LDA), and feature-based local binary pattern (LBP).