Online stereo calibration using FPGAs

Online stereo calibration is useful in many situations where the cameras are moving relative to each other. The motion can either be intentional, as in an active stereo head, or due to vibrations, heat etc. which is commonly found in automotive applications. However, most approaches for finding the essential matrix relating the two cameras are computationally very expensive and, hence, this problem must be addressed. In this paper, we suggest deferring a large portion of the image processing onto a field programmable gate array (FPGA) since most operations can be heavily parallelized. The specific algorithm chosen to find point correspondences between the left and the right images is SIFT, which has the advantage of producing a very small number of outliers. Having few outliers is important as computing the essential matrix from point correspondences is an inherently unstable problem, particularly in the case where the cameras are nearly parallel. We present a system, which computes the computationally intensive parts of SIFT (Gaussian pyramid, Sobel etc) using an FPGA. The host computer then uses the resulting point correspondences to estimate the essential matrix with the help of a reduced model of the camera setup. On-line stereo calibration at frame rate (60Hz) is then possible without excessively loading the host computer.