Recognition of occluded facial expressions using a Fusion of Localized Sparse Representation Classifiers

Facial expression recognition (FER) methods that are localized have a distinct advantage over whole face methods in situations where part of the face is occluded. We present a new algorithm that first classifies local regions of the face around each eye and the mouth using a Sparse Representation Classifier (SRC). Then, it dynamically determines the set of local regions that best classifies the facial expression by using the representation error characteristics from each local region. This algorithm is termed Fusion of Local Sparse Representation Classifiers (FLSRC). In an experiment where the mouth region is occluded, the FLSRC recognition rate was 93.4% compared to 72.8% using SRC with all available pixels and rates <; 60% using Gabor or PCA based methods. FLSRC significantly outperforms other methods when large facial regions are occluded. For example, when approximately 25% of the image is occluded, FLSRC gives a recognition rate of 85% compared to <; 65% for other methods.