PCA-based structure refinement for reconstruction of urban scene

There is plenty of structured information (such as lines and planes) in urban scenes. Considering this, we propose a new method for making use of this information to enhance the reconstruction of urban outdoor scenes. Structured information (collinearity and coplanarity) is extracted from images by performing line detection and color image segmentation, which is used as hypothetic constraints of the 3D structure. In refining stage, we first build PCA subspaces for each structured components (collinear and coplanar point sets), during which the former hypothetic structure information is further inspected by the initial 3D structure. Then we iteratively update the structure through EM estimation. Experiments show that this method effectively improves the accuracy and robustness of reconstruction of urban scenes.