Image-based 3D surface reconstruction by combination of photometric, geometric, and real-aperture methods

An image-based 3D surface reconstruction method based on simultaneous evaluation of intensity and polarisation features (shape from photopolarimetric reflectance) and its combination with absolute depth data is introduced in this article. The proposed technique is based on the analysis of single or multiple intensity and polarisation images. To compute the surface gradients, we present a global optimisation method based on a variational framework and a local optimisation method based on solving a set of non-linear equations individually for each image pixel. These approaches are suitable for strongly non-Lambertian surfaces and those of diffuse reflectance behaviour and can also be adapted to surfaces of non-uniform albedo. We describe how independently measured absolute depth data is integrated into the shape from photopolarimetric reflectance framework in order to increase the accuracy of the 3D reconstruction result. In this context we concentrate on dense but noisy depth data obtained by depth from defocus and on sparse but accurate depth data obtained by stereo or structure from motion analysis. We show that depth from defocus information should preferentially be used for initialising the optimisation schemes for the surface gradients. For integration of sparse depth information, we suggest an optimisation scheme that simultaneously adapts the surface gradients to the measured intensity and polarisation data and to the surface slopes implied by depth differences between pairs of depth points. In principle, arbitrary sources of depth information are possible in the presented framework. Experiments on synthetic and on real-world data reveal that while depth from defocus is especially helpful for providing an initial estimate of the surface gradients and the albedo in the absence of a-priori knowledge, integration of stereo or structure from motion information significantly increases the 3D reconstruction accuracy. In our real-world experiments, we regard the scenarios of 3D reconstruction of raw forged iron surfaces in the domain of industrial quality inspection and the generation of a digital elevation model of a section of the lunar surface in the context of space-based planetary exploration.