Surface reconstruction by integrating 3D and 2D data of multiple views

Surface representation is needed for almost all modeling and visualization applications, but unfortunately, 3D data from a passive vision system are often insufficient for a traditional surface reconstruction technique that is designed for densely scanned 3D point data. In this paper, we develop a new method for surface reconstruction by combining both 3D data and 2D image information. The silhouette information extracted from 2D images can also be integrated as an option if it is available. The new method is a variational approach with a new functional integrating 3D stereo data with 2D image information. This gives a more robust approach than existing methods using only pure 2D information or 3D stereo data. We also propose a bounded regularization method to implement efficiently the surface evolution by level-set methods. The properties of the algorithms are discussed, proved for some cases, and empirically demonstrated through intensive experiments on real sequences.