Hand posture recognition using Hidden Conditional Random Fields

Body-language understanding is essential to human robot interaction, and hand posture recognition is one of the most important components in a body-language recognition system. The existing hand posture recognition approaches based on robust local features such as SIFT can be invariant to background noise and in-plane rotation. However the ignorance of the relationships among local features is a fundamental issue. The part-based models argue that objects of the same category share the same part-structure which consists of parts and relationships among parts. In this paper, a discriminative part-based model, hidden conditional random fields (HCRFs), is used to recognize hand postures. Although the existing global locations of features have been used to consider large scale dependency among parts in the HCRFs framework, the results are not invariant to in-plane rotation. New features by the distance to the image center are proposed to encode the global relationship as well as to perform in-plane rotation-invariant recognition. The experimental results demonstrate that the proposed approach is in-plane rotation-invariant and outperforms the approach using AdaBoost with SIFT.