Spatio-temporal feature extraction-based hand gesture recognition for isolated American Sign Language and Arabic numbers

This paper proposes a system to recognize isolated American Sign Language and Arabic numbers in real-time from stereo color image sequences using Hidden Markov Models (HMMs). Our system is based on three main stages; preprocessing, feature extraction and classification. In preprocessing stage, color and 3D depth map are used to detect and track the hand. The second stage, 3D combined features of location, orientation and velocity with respected to Cartesian and Polar systems are used. Additionally, k-means clustering is employed for HMMs code-word. In the final stage, the hand gesture path is recognized using Left-Right Banded topology (LRB) in conjunction Viterbi path. Experimental results demonstrate that, our system can successfully recognize isolated hand gestures with 98.33% recognition rate.