Facial emotion detection using GPSO and Lucas-Kanade algorithms

Emotion detection is receiving a lot of attention from researchers due to its potentials in improving human-computer interaction. Recently, we proposed a modification to the Particle Swarm Optimization (PSO) algorithm for the purpose applying it to emotion detection. Our algorithm, which we called Guided Particle Swarm Optimization (GPSO), involves studying the movements of specific points, called action units (AUs), placed on the face of a subject, as the subject expresses different emotions. A swarm of particles is defined such that each particle consists of components from the neighborhood of each AU. However, instead of applying the pure PSO on the swarm to detect emotions, we made the algorithm to take into account the positions of the AUs - thus, the swarm is effectively guided to converge on the path of the AUs. We showed this approach to work very well and made the swarm to converge very quickly to identify the emotion being expressed. One limitation to our earlier system was that the AUs must be physically specified on the subject before the video clips are recorded. In this paper, we present an improvement on the system where we specify the AUs at runtime in a video stream and then apply LK algorithm to keep track of their positions, thus making the system to work on real time basis with the same promising detection success rates. Potential application areas of our system include medical engineering, forensic applications by police and psychiatric applications.