Robust piecewise planar stereo with modified segmentation cues in urban scenes

This paper proposes a robust piecewise planar multi view stereo (MVS) approach specifically designed for urban scenes. These architectural scenes are problematic for traditional computer vision methods. In our work, we focus on exploiting some useful constraints of artificial structures such as piecewise coplanarity and boundaries of superpixels. Firstly, we reconstruct quasi-dense 3D point clouds of urban scenes using patches-based MVS (PMVS) method. Secondly, a set of 3D candidate planes are generated by the obtained point clouds without any assumption on the normals of planes, unlike famous Manhattan-world assumption. Then, we segment multi-view images with watershed algorithm and modify the contours of superpixels by the classical Douglas-Peucker approximation algorithm to fit the contours to the boundaries of objects in urban scenes as much as possible. Finally, we use the candidate planes as labels and superpixels as nodes to formulate our Markov Random Field (MRF) optimization problem, then a piecewise planar depth map for each view is recovered by solving the optimization problem using graph-cuts. Experiments show that our method outperforms previous approaches in terms of accuracy.