Temporal integration of multiple silhouette-based body-part hypotheses

A method for temporally integrating appearance-based body-part labelling is presented. We begin by modifying the silhouette labelling method of Ghost (Haritaoglu, Harwood, and Davis. 1998); that system first determines which posture best describes the person currently and then uses posture-specific heuristics to generate labels for head, hands, and feet. Our approach is to assign a posture probability and then estimate body part locations for all possible postures. Next we temporally integrate these estimates by finding a best path through the posture-time lattice. A density-sampling propagation approach is used that allows us to model the multiple hypotheses resulting from consideration of different postures. We show quantitative and qualitative results where the temporal integration solution improves the instantaneous estimates. This method can be applied to any system that inherently has multiple methods of asserting instantaneous properties but from which a temporally coherent interpretation is desired.