Belief Propagation Driven Method for Facial Gestures Recognition in Presence of Occlusions

In this paper, we present a framework for automatic facial gestures recognition in a highly interactive environment. We propose to use a region-based representation of the face, and model the interdependencies of these regions using a graphical model. A parametric modeling of the local facial deformations and the use of a graphical model facilitate the characterization of intra-region dynamic patterns and the inter-region dependencies. We augment this formalism with a belief propagation algorithm to infer missing data, as well as correcting for erroneous estimations of the local deformations. The resulting approach hence handles complete observation and partial occlusion cases in an unified way, and allows for the recognition of facial gestures in presence of head motion and partial occlusions of the face. Experimental results on recognizing facial expression in various real world situations, such as non-frontal views, moving head, and occlusions illustrate the proposed approach.