From Joyous to Clinically Depressed: Mood Detection Using Multimodal Analysis of a Person´s Appearance and Speech

Clinical depression is a critical public health problem, with high costs associated to a person´s functioning, mortality, and social relationships, as well as the economy overall. Currently, there is no dedicated objective method to diagnose depression. Rather, its diagnosis depends on patient self-report and the clinician´s observation, risking a range of subjective biases. Our aim is to develop an objective affective sensing system that supports clinicians in their diagnosis and monitoring of clinical depression. In this PhD work, my approach is based on multimodal analysis, i.e. combinations of vocal affect, head pose and eye movement from a video-audio real-world clinically validated data. In addition, this work will investigate the cross-cultural generalization of depression characteristics from different languages and countries.