Object Centered Stereo: Displacement Map Estimation Using Texture and Shading

We consider the problem of recovering 3D surface displacements using both shading and multi-view stereo cues. In contrast to traditional disparity or depth map representations, the object centered displacement map representation enables the recovery of complete 3D objects while also ensuring the reconstruction is not biased towards a particular image. Although displacement mapping requires a base surface, this base mesh is easily obtained using traditional computer vision techniques (e.g., shape-from-silhouette or structure-from-motion). Our method exploits shading variation due to object rotation relative to the light source, allowing the recovery of displacements in both textured and textureless regions in a common framework. In particular, shading cues are integrated into a multi-view stereo photo-consistency function through the surface normals that are implied by the displacement map. The analytic gradient of this photo-consistency function is used to drive a multi-resolution conjugate gradient optimization. We demonstrate the geometric quality of the reconstructed displacements on several example objects including a human face.