Real Time Rectification for Stereo Correspondence

Duplicating the full dynamic capabilities of the human eye-brain combination is a difficult task but an important goal because of the wide application for a system which can acquire accurate 3D models of a scene in real time. Such a system must be able to correct images to remove lens distortion and camera misalignments from high resolution images at video frame rates 30 fps or better. The images then need to be matched to determine the distance to scene objects. We have constructed a system which uses reconfigurable hardware (FPGAs) to handle the very large number of calculations required and is capable of processing 1 Mpixel images for disparity ranges of ~ 100 (allowing ~ 1% depth accuracy) at 30 fps. This paper focuses on the use of lookup tables in the hardware to correct images in real time with latencies that are determined more by the quality of the optics and mechanical alignment than by calculation demand. Sample results from the full system (which uses the Symmetric Dynamic Programming Stereo matching algorithm) operating at 30 fps are also shown.