Stereo vision based video real-time 3D pointing gesture recognition

Among natural human gestures occurring during non-verbal communication, pointing gesture can be easily recognized and taken as more natural new human computer interfaces. An approach to pointing gestures recognition is proposed based on binocular stereo vision. A robust automatic foreground extraction algorithm is developed based on wavelet multi-scale transformation across background subtraction without the need to determine an optimal threshold for each image with different clutter background. The method provides 3D information regarding the tracked hands and head. During solving the correspondence of fingertip of the pointing arm, a stereo matching strategy is presented based on geometric constraints between fingertip points, arm axis and epipolar line. Continuity constraint is adopted to insure the validity of the processed correspondence. Experimental results show that the system works in real time on a low cost hardware platform. It is fairly accurate and independent of user characteristics and environmental changes.