Intraindividual and interindividual multimodal emotion analyses in Human-Machine-Interaction

Interactive computer systems today interact nearly effortlessly with humans by menu-driven mouse- and text-based input. In case of other modalities like audio and gesture control systems still lack on flexibility. To respond appropriately, these intelligent systems require specific cues about the user´s internal state. Reliable emotion recognition of technical systems is therefore an important issue in computer sciences and applications. In order to develop an appropriate methodology for emotion analyses, a multimodal study is introduced here. Audio and video signals as well as biopsychological signals of the user are applied to detect intraindividual behavioural prototypes that can be used for predictions of the user´s emotional states. Additionally, interindividual differences are considered and discussed. Statistical analyses showed results in most cases with statistical significance of probability value p <; 0.05 and an effect size d >; 1.05.