Multi-view photometric stereo of non-Lambertian surface under general illuminations

We present an approach to reconstruct 3D fine-scale surface models for non-Lambertian objects from multi-view multi-illumination image sets. Unlike most previous work in photometric stereo, this approach works for general lighting conditions, i.e. natural outdoor illumination. Our method begins with a raw 3D model reconstructed from available multi-view stereo techniques. Considering the sparse characteristics of surface reflectance in the view-illumination space, we first estimate the diffuse appearance of the 3D model from the multiview captured images, and then refine it using the surface appearance under varying illuminations. With the separated low rank diffuse component, we exploit the photometric cues to recover detailed surface structure. Experimental results on various real world scenes validate that the proposed method is able to handle surfaces with specular reflectance even including saturated colours, highlight and cast-shadows.