Mutual information between inter-hemispheric EEG spectro-temporal patterns: A new feature for automated affect recognition

Automated electroencephalography (EEG) based affect recognition has gained a lot of interest recently, with clinical (e.g., in autism), human-computer interaction (e.g., affective brain-computer interfaces), neuromarketing, and even multimedia (e.g., affective video tagging) applications. Typically, conventional EEG features such as spectral power, coherence, and frontal asymmetry have been used to characterize affective states. Recently, cross-frequency coupling measures have also been explored. In this paper, we propose a new feature set that combines some of these aforementioned paradigms. First, the full-band EEG signal is decomposed into four subband signals, namely theta, alpha, beta, and gamma. The amplitude modulation (or envelope) of these signals is then computed via a Hilbert transform. These amplitude modulations are further decomposed into 10 cross-frequency coupling patterns (e.g., gamma-beta coupling pattern). The mutual information between each of these ten patterns is then calculated for all inter-hemispheric EEG electrode pairs. To gauge the effectiveness of the newly-proposed feature set, the so-called DEAP database was used. Experimental results show the proposed feature set outperforming conventional ones for estimation of arousal, valence, dominance, and liking affective dimensions. Gains of up to 20% could be achieved when the proposed features were fused with spectral power and asymmetry index features, thus suggesting complementarity between spectral and spectro-temporal features for automated affective state recognition.