Automatic Single View Building Reconstruction by Integrating Segmentation

In this paper, we propose a stochastic algorithm using Markov chain Monte Carlo (MCMC) to automatically reconstruct buildings from a single image of architectural scenes by integrating segmentation and reconstruction. Buildings are modelled by two families of generative models: One is parameterized geometric primitives (e.g. boxes, prisms) for 3D structures of buildings. The other is image models for appearances of building surfaces. All the other objects except buildings in the scene are modelled as a 3D background plane with some appearance. Regarding one image of architectural scenes as the 2D projection of the appearances of all the component primitives in the buildings and the background plane to the image plane, we reconstruct buildings under the Bayesian statistical framework by inferring the 3D structure of its component primitives and image models of visible surfaces, which follow some spatial relation prior and reproduce the given image under some projection matrix. The aspect hierarchy is used to generate proposals in primitive space, which can greatly speed up the Markov chain search.