Facial expression recognition using thermal image processing and neural network

A method using thermal image processing and a neural network is presented for recognition of facial expression. The method is based on 2-dimensional detection of temperature distribution of the face, using infrared rays. The front-view face in the input image is normalized in terms of the size and the location, followed by measurement of the local temperature difference between the averaged neutral and the unknown expression faces. For measuring the local temperature difference, several blocks on the face are selected, based on the studies in the fields of psychology, especially FACS-AU, and computer vision with visible ray, while the blocks including cheekbone region are selected experimentally for improving recognition accuracy. The local temperature difference caused by the rearrangement of face muscle and the inner temperature change is used as input data for a neural network. By using the backpropagation method, neutral, happy, surprised and sad expressions were recognized with 90% accuracy