A new minimal solution to the relative pose of a calibrated stereo camera with small field of view overlap

In this paper we present a new minimal solver for the relative pose of a calibrated stereo camera (i.e. a pair of rigidly mounted cameras). Our method is based on the fact that a feature visible in all four images (two image pairs acquired at two points in time) constrains the relative pose of the second stereo camera to lie on a sphere around this feature, which has a known, triangulated position in the first stereo camera coordinate frame. This constraint leaves three degrees of freedom; two for the location of the second camera on the sphere, and the third for the rotation in the respective tangent plane. We use three 2D correspondences, in particular two correspondences from the left (or right) camera and one correspondence from the other camera, to solve for these three remaining degrees of freedom. This approach is amenable to stereo cameras having a small overlap in their views. We present an efficient solution for this novel relative pose problem, describe the incorporation of our proposed solver into the RANSAC framework, evaluate its performance given noise and outliers, and demonstrate its use in a real-time structure from motion system.