Hand posture estimation by combining 2-D appearance-based and 3-D model-based approaches

This paper proposes a method for the rapid and precise estimation of human hand postures by combining 2D appearance matching and 3D model-based fitting. First a rough posture estimate is obtained by image indexing. Each possible hand appearance generated from a given 3D shape model is labeled by an index obtained by PCA compression and registered with its 3D model parameters in advance. By retrieving the index of the input image, the method can obtain the matched appearance image and its 3D parameters rapidly. Then, starting from the obtained rough estimate, it estimates the posture and moreover refines the given initial 3D model by model-fitting. The sequential activation of the two processes in every frame gives the precise posture estimate rapidly. The effectiveness of the method is shown by experimental results