Aligning sequences from multiple cameras

This paper studies the problem of aligning images from multiple cameras with minimally- or non-overlapping fields of view using frame-to-frame transformations calculated for sequences from each camera. We examine implementation issues for the algorithm of Caspi and Irani that performs the alignment for two cameras which are fixed relative to each other and have approximately the same center of projection. Furthermore, we extend it to compute the camera-to-camera homographies for an N≥2 multi-camera network by simultaneously solving for all parameters of the unknown transformations. This enforces tighter constraints on the solution than performing the alignment for each pair independently. We show the efficacy of the approach on both synthetic as well as real sequences captured using a polycamera built in our lab. The aligned images can be mosaiced together to obtain a wider field of view virtual camera for subsequent processing.