Heterogeneous feature code for expression recognition

The Heterogeneous Feature Code (HFC), a coding scheme based on both human and machine selected local features, is proposed for expression recognition. The HFC consists of two component codes, the Human Observable Code (HOC) and Boost Feature Code (BFC). The HOC is developed to capture the local deformation patches observable to humans when the face is showing an expression. Different expressions appear with a specific set of such patches with different deformation patterns at different locations, which are considered in the configuration of the HOC codewords. The BFC is built upon the mutually connected Haar-like features selected by a set of Adaboost classifiers followed by a multi-class SVM classifier. Unlike the HOC features, the BFC features can hardly be selected by human eyes. The HFC is probably the first code that combines human selected features and machine selected features, and proven effective for expression recognition. Performance evaluation on the Cohn-Kanade extension (CK+) database and the Japanese Female Facial Expression (JAFFE) shows that the HFC outperforms either HOC or BFC component code alone, and is competitive to the state-of-the-art.