A framework for the integration of gesture and posture recognition using HMM and SVM

For a successful real-time vision-based HCI system, inference from natural visual method is crucial. In this paper, we have aimed to provide interaction through gesture and posture recognition for alphabets and numbers. In addition, data fusion is carried out which integrates these systems to extract multiple meanings at the same time. 3D information is exploited for segmentation and detection of face and hands using normal Gaussian distribution and depth information. For gesture, orientation of two consecutive hand centroid points is computed which is then quantized to generate code words. HMM is trained by Baum Welch algorithm and classified by Viterbi path algorithm. In posture recognition, American Sign Language is recognized for static alphabets and numbers. Feature vectors are computed from statistical and geometrical properties of the hand and are used to train SVM for classification and recognition. Moreover, curvature analysis is carried out for alphabets to avoid misclassifications. Experimental results of the proposed framework successfully integrate both gesture and posture recognition system at decision level fusion whereas the gesture system achieves recognition rate of 98% (i.e. for alphabets and numbers) and the posture recognition system with recognition rates of 98.65% and 98.6% for ASL alphabets and numbers respectively.