Multimodal Voxelization and Kinematically Constrained Gaussian Mixture Models for Full Hand Pose Estimation: An Integrated Systems Approach

We propose a new approach to the hand pose estimation problem using only volume information. We describe a thermal and color image-based approach to generate silhouettes of the hand with which voxel images are produced using shape-from-silhouette. We assume a 16-component, 27 degree-of-freedom kinematically constrained Gaussian mixture model, and fit it over the voxel images. We constrain the otherwise freely-arranged components of this model by a system of equations describing joint characteristics parameterized by component centroids and orientations. We use the EM algorithm and steepest descent to estimate the parameters of the model such that they yield the maximum likelihood and kinematically correct pose estimates. We demonstrate the effectiveness of the proposed system on synthesized as well as captured voxel images of the hand, and show that given appropriate initial conditions, the iterative model parameter estimation procedure effectively converges to and tracks the "voxelized" articulated hand.