Human emotion recognition: An interval type-2 fuzzy inference system based approach

In this paper, an Interval Type-2 neuro-fuzzy inference system based emotion recognition system is proposed. The employed fuzzy inference system is a four layer network realizing Takagi-Sugeno-Kang fuzzy inference mechanism, with an input layer, a rule layer, a normalization layer and an output layer. The rule layer employs an Interval Type-2 fuzzy membership function to handle the uncertainty in the facial emotions of different individuals. The rules for this network is generated by employing a meta-cognitive projection based learning algorithm. The aim of the proposed approach is to approximate the decision surface separating different emotions based on noisy input features. During learning, as a sample is presented to the network, it calculates the prediction error and knowledge content in the sample to decide on whether to learn the sample, when to learn it and which technique to use to learn it. The meta-cognitive learning mechanism helps in avoiding overtraining and achieving better generalization performance. The projection based learning approach employed to learn the knowledge in a given sample works by minimizing the total energy in the network in a linear least squared sense. The proposed emotion recognition system is tested on two well-known publicly available datasets: Japanese female facial expression dataset and Taiwanese female expression image database. Local binary pattern based features are extracted from these databases, as they have been shown to describe facial features such as edges and spots, efficiently. Two different studies are performed: a 5-fold cross-validation study on the emotion recognition ability of the system and a database independent study. The performance comparison with other approaches clearly highlights the advantage of the proposed system.