3D Surface Reconstruction from Point-and-Line Cloud

We present a method for reconstructing 3D surface as triangular meshes from imagery. The surface reconstruction requires 3D point cloud for composing vertices of triangle meshes. A standard approach uses incremental structure from motion (SfM) to obtain camera poses and sparse 3D point cloud that are given based on 2D key-point matching. As the 3D surface directly reconstructed from the sparse 3D point cloud often lack detail of objects, multiple-view stereo (MVS) is commonly used to generate dense 3D point cloud. A known problem with the densification is that MVS generates many small patches even for planar flat objects that degrade the quality of surface model. Using dense 3D point cloud also requires high memory capacity for visualization. In this work, we propose to reconstruct 3D surface using sparse 3D point cloud generated by SfM and 3D line segments (3D line cloud) computed from multiple views since these two elements can complement well for representing man-made structures. The proposed method extends the tetrahedra-carving method as it can use 3D point-and-line cloud under the global optimization framework. We demonstrate that the proposed method can efficiently produce surface models whose quality are at least as good as the baseline method using dense 3D point cloud.