HMM-Based Deception Recognition from Visual Cues

Behavioral indicators of deception and behavioral state are extremely difficult for humans to analyze. This research effort attempts to leverage automated systems to augment humans in detecting deception by analyzing nonverbal behavior on video. By tracking faces and hands of an individual, it is anticipated that objective behavioral indicators of deception can be isolated, extracted and synthesized to create a more accurate means for detecting human deception. Blob analysis, a method for analyzing the movement of the head and hands based on the identification of skin color is presented. A proof-of-concept study is presented that uses Blob analysis to extract visual cues and events, throughout the examined videos. The integration of these cues is done using a hierarchical hidden Markov model to explore behavioral state identification in the detection of deception, mainly involving the detection of agitated and over-controlled behaviors