A unified rectification method for single viewpoint multi-camera system

Stereo matching and 3D reconstruction has been studied for decades as a fundamental problem in the field of computer vision. Recent years, stereo matching and 3D reconstruction with a large field of view, especially using omnidirectional vision and panoramic images, have received increasing attention. As a pre-step for dense stereo matching, methods are proposed to rectify different kinds of omnidirectional stereo image pairs. However, no one has described a rectification method applied to multi-camera omnidirectional systems yet. In this work, we proposed a rectification algorithm based on spherical camera model for rectifying omnidirectional stereo pairs, especially well suitable for the multi-camera omnidirectional systems as long as a spherical camera model is able to be applied. We describe the geometrical framework of the algorithm and implement it. Also we present the experimental results of the real stereo image pairs captured by Ladybug3. As the experimental results show, the effect of rectification is promising.