Scene flow constrained multi-prior patch-sweeping for real-time upper body 3D reconstruction

We present a high performance real-time approach for robust 3D structure estimation of human bodies. Our idea extends state of the art high precision patch-sweep techniques by exploiting temporal information. By parameterizing 3D patches in the spatio-temporal domain we gain increased quality and robustness, while the computational complexity decreases drastically. The key to these improvements is the utilization of a global scene flow like prior called spatio-temporal object (STO). The target application of our method reaches from video communication and virtual eye contact scenarios till future digital cinema 3D post-production and real-time person modelling and modification. The result of this work extends a novel 3D scene representation developed in the EC research project SCENE.