Stereo mosaics with slanting parallel projections from many cameras or a moving camera

This paper presents an approach of fusing images from many video cameras or a moving video camera with external orientation data (e.g. GPS and INS data) into a few mosaicked images that preserve 3D information. In both cases, a virtual 2D array of cameras with FOV overlaps is formed to generate the whole coverage of a scene (or an object). We propose a representation that can re-organize the original perspective images into a set of parallel projections with different slanting viewing angles. In addition to providing a wide field of view, there are two more benefits of such a representation. First, mosaics with different slanting views represent occlusions encountered in a usual nadir view. Second, stereo pair can be formed from a pair of slanting parallel mosaics thus image-based 3D viewing can be achieved. This representation can be used as both an advanced video interface for surveillance or a pre-processing for 3D reconstruction.