Person appearance modeling and orientation estimation using Spherical Harmonics

We present a novel approach for the joint estimation of a person´s overall body orientation, 3D shape and texture, from overlapping cameras. Overall body orientation (i.e. rotation around torso major axis) is estimated by minimizing the difference between a learned texture model in a canonical orientation and a texture sampled using the current 3D shape estimate (i.e. torso and head). The estimated body orientation subsequently allows to update the 3D shape estimate, taking into account the new 3D shape measurement obtained by volume carving. Our main contribution is a method for estimating a person´s relative body orientation while simultaneously generating a basic Spherical Harmonics based model of the person´s shape and texture. Experiments show that the proposed method outperforms two state-of-the-art orientation estimation methods: one combining a fixed 3D shape model with a generate-and-test texture matching approach and one using a classifier based approach.