Stereo correspondence with slanted surfaces: critical implications of horizontal slant

We examine the stereo correspondence problem in the presence of slanted scene surfaces. In particular we highlight a previously overlooked geometric fact: a horizontally slanted surface (i.e. having depth variation in the direction of the separation of the two cameras) will appear horizontally stretched in one image as compared to the other image. Thus, while corresponding two images, N pixels on a scanline in one image may correspond to a different number of pixels M in the other image. This leads to three important modifications to existing stereo algorithms: (a) due to unequal sampling, intensity matching metrics such as the popular Birchfield-Tomasi procedure must be modified, (b) unequal numbers of pixels in the two images must be allowed to correspond to each other, and (c) the uniqueness constraint, which is often used for detecting occlusions, must be changed to a 3D uniqueness constraint. This paper discusses these new constraints and provides a simple scanline based matching algorithm for illustration. We experimentally demonstrate test cases where existing algorithms fail, and how the incorporation of these new constraints provides correct results. Experimental comparisons of the scanline based algorithm with standard data sets are also provided.