Semi-dense Piecewise Planar Stereo Reconstruction Using SymStereo and PEARL

We propose an algorithm for two-frame stereo reconstruction that generates an accurate semi-dense piecewise planar model of the 3D scene. First, we use the SymStereo framework [1], [2] for obtaining a semi-dense stereo reconstruction, which exploits symmetry cues for estimating depth along virtual cut planes. Next, the energy-based multi-model fitting algorithm PEARL [3] is applied for estimating the plane models, and labeling the lines of intersection between the virtual and epipolar planes. This procedure involves three steps: (i) generate an initial set of plane hypotheses by extracting line cuts from each virtual cut plane, (ii) inlier classification, and (iii) a plane model refinement step. We provide experiments of our approach on a variety of challenging indoor and outdoor stereo pairs. The results show that we successfully detect and reconstruct the planar surfaces contained in the scene, as well as are able to obtain plausible piecewise planar descriptions of non-planar objects. In contrast to existing approaches, the accuracy of our plane hypotheses are not limited by an initial 3D reconstruction, but are iteratively refined after each plane labeling.