Classifying emotions in rehabilitation robotics based on facial skin temperature

Rehabilitation robotic plays an important role in therapeutic exercises by combining robots with computer serious games into an attractive therapeutic platform. However, measuring the degree of engagement of the user is not a trivial task. The difficulty of applying question-based techniques, particularly for patients who have the speech capacity compromised due to cerebrovascular accidents, has inspired us to investigate noninvasive and nonverbal techniques aiming to classifying emotions. For this purpose, a supervised artificial neural network interprets facial infrared thermal images of individuals performing rehabilitation robotic therapy integrated with games. A database containing images of 8 users was generated by combining evoked and spontaneous emotional reactions. In total, 2445 facial thermal images with an average of 100 images per person for three categories of emotions (neutral, motivated, overstressed) were classified. Based on confusion matrix analysis, the experimental results correlated very well with manual estimates, producing an overall performance of 92.6%.