Real-Time Hand Tracking and Gesture Recognition Using Semantic-Probabilistic Network

In this paper we propose a system for real-time hand gesture recognition for use as a human-computer interface. Firstly, hand detection is performed using a Viola-Jones algorithm. We use the Continuously Adaptive Mean Shift Algorithm (CAM Shift) to track the position of each detected hand in each video frame. The hand contour is then extracted using a Border Following algorithm, which is preceded by skin-colour thresholding, performed in HSV colour space. Finally, a semantic-probabilistic network is introduced, which uses an ontological gesture model and Bayesian network for gesture recognition. We then demonstrate the effectiveness of our technique on a training data set, as well as on a real-life video sequence.