A markerless motion capture system with automatic subject-specific body model acquisition and robust pose tracking from 3D data

We present a markerless system for recovering 3D full-body motion from multiple image sequences. Our approach supports robust pose tracking from 3D data as well as automatic body model acquisition. For initialization, a subject-specific voxel body model that fits well to the shape of the subject being tracked is automatically created from the beginning volume. Then for pose recovery, the voxel body model is matched to the image features via a hierarchical pose search method. We use a new particle based stochastic search algorithm and introduce a robust metric, which is incorporated with joint limits, physical constraints and fuses multiple 3D cues involving volume spatial and 3D scene flow motion information. Results on several complex multiple sequences show the robustness and effectiveness of our approach.