Facial Expression Recognition from Image Sequences Based on Feature Points and Canonical Correlations

In this paper, we present a framework to recognize facial expressions from image sequences. Our method uses optical flow to track feature points in sequential facial frames, and computes normalized displacements of key feature points and certain standardized geometric distances to form a matrix, called facial-expression-arising-dataset (FEAD). Each FEAD represents an expression image sequence (from neutral to peak). We use canonical correlations to classify an FEAD into one of the six basic facial expressions, and utilize a linear discriminant function to optimize the learning and recognition process. Our method formulates the facial expression recognition as data sets matching problem to fully utilize the dynamic information in expression emerging process, and achieves a recognition accuracy of beyond 90%. Experimental results demonstrate the robustness and effectiveness of this method.