Towards emotion aware computing: A study of arousal modulation with multichannel event-related potentials, delta oscillatory activity and skin conductivity responses

Emotion identification has recently been considered as a key element in contemporary studies for advanced human-computer interaction. The achievement of this goal is usually attempted via methods incorporating facial expression and speech recognition, as well as, human motion analysis. In this paper it is attempted to fuse multi-modal physiological signals of the autonomic (skin conductance) and central nervous systems (EEG), through the use of appropriate feature extraction procedures discriminating emotional arousal modulations, to a neural network classifier. Thus, skin conductivity responses, evoked-related potential peaks, and delta frequency oscillatory patterns are analyzed for a comparatively large number of subjects exposed to different emotions, evoked by pictures selected from the International Affective Picture System. The achieved neural network classifications were encouraging. It was found that fear was successfully differentiated (100%), pleasant emotions differing in their arousal level were well distinguished (80%), but the discrimination of low arousing negative feelings such as melancholy was more difficult (70%). It is argued that physiological patterning of multimodal recordings may successfully contribute to the enhancement of human computer interaction and emotion aware computing.