Part template: 3D representation for multiview human pose estimation

We present a system for human pose estimation from synchronized multiview images. The system uses an analysis-by-synthesis approach with a skeleton model. This approach is powerful, but may present issues with its potentially huge search space. We adopt a hierarchical method where the head and torso are found first based on template fitting. The detection of the other parts then proceeds with the shoulders and hips to locate the anchor points of the limbs. Subsequently, a hierarchical fitting technique is used to estimate the location of the limbs. The parameter space is then partitioned, which dramatically reduces the complexity of pose estimation. Another difficulty of this system is to find adequate measurements which are used to fit the skeleton model. A multi-cue 3D fusion method is proposed for this purpose. It starts with extracting a set of cues from synchronized multiview images which exploit geometric and color information, and they are then integrated into a 3D representation, called a “part template”. The experiments show that this system reliably performs on sequences that include unconstrained motions, such as those that are fast or unpredictable, and is also robust to several common issues associated with input data, such as image noise and self-contact.