Facial Expression Recognition Using Deep Boltzmann Machine from Thermal Infrared Images

Facial expression recognition from thermal infrared images has attracted more and more attentions in recent years. However, the features adopted in current work are either temperature statistical parameters extracted from the facial regions of interest or several hand-crafted features that are commonly used in visible spectrum. Till now there is no image features specially defined for thermal infrared images. In this paper, we are the first to propose using the Deep Boltzmann Machine to learn thermal features for expression recognition from thermal long wavelength infrared images. First, the face are located and normalized from the thermal infrared images. Then, a Deep Boltzmann Machine model composed of two layers is proposed. The parameters of the Deep Boltzmann Machine model are further fine-tuned for facial expression recognition after pre-training of feature learning. Comparison experimental results on the NVIE database demonstrate that our approach outperforms other approaches using temperature statistic features or hand-crafted features borrowed from visible domain. The learned features from the forehead, mouth, and cheek are more reliable for discriminating disgust, fear, and happiness compared with other facial areas.