Fuzzy-GIST for 4-emotion recognition in natural scene images

In this paper we propose a novel “fuzzy-GIST” for analyzing the subject specific emotion reflected by a natural scene, considering both the human emotional state and the visual features extracted from the scene image. According to the relationship between emotional factors and the characters of image, we incorporate the fuzzy concept to extract emotional features using L*C*H* color and orientation information. On the other hand, after various pre-processing of emotional electroencephalography (EEG), we treat emotional relevant EEG features using the fuzzy logic based on possibility theory rather than widely used conventional probability theory to generate the semantic feature of the human emotions. Fuzzy-GIST consists of both semantic visual information and linguistic EEG feature, it is used to represent emotional gist of a natural scene in a semantic level. We use a neuro-fuzzy inference model to infer the emotion evoked by an image, and the feedback from the subject is used for supervising the learning as well as evaluating the performance of the proposed scheme. The experiment results show the possibility to recognize four different emotions for a given dataset. Moreover, the analysis of human emotional status and visual information also can serve for the study of interaction between human subject and machine, and possibly the development of new brain machine interface.