Vision-based hand gesture tracking and recognition

Summary form only given. We present some results of our research on hand tracking and gesture recognition in the last 5 years. This research is motivated by applications in human-computer interaction such as display control in virtual environments and the manipulation of virtual objects. Although we have studied both appearance-based and 3D model-based approaches, this talk concentrates on the latter. At any given time instant, the hand configuration - 6 parameters for the global hand "pose" and 21 finger joint angles (the hand "posture") - is a point the 27-dimensional configuration space. We track the trajectory of this point over time using a 3D model-based and analysis-by-synthesis approach. The challenge is to represent the constraints on hand posture and finger movement in a compact way and use this representation to speed up the search in the 21-dimensional space. The tracking results can then be used to do gesture recognition. Some preliminary results are shown on constraint representation and its use in tracking.