FPGA-based rectification of stereo images

In order to obtain depth information about a scene in computer vision, one needs to process pairs of stereo images. The calculation of dense depth maps in real-time is computationally challenging as it requires searching for matches between objects in both images. The task is significantly simplified if the images are rectified, a process which horizontally aligns the objects in both images. The process of stereo images rectification has several steps with different computational requirements. The steps include 2D searches for high fidelity matches, precise matrix calculations, and fast pixel coordinate transformations and interpolations. In this project, the complete process is effectively implemented in a Spartan-3 FPGA, taking advantage of a MicroBlaze soft core for slow but precise calculations, and of fast dedicated hardware support for achieving the real-time requirements. The implemented system successfully performs real-time rectification on the images from two video cameras, with a resolution of 640×480 pixels and a frame rate of 25 fps, and is easily configured for videos with higher resolutions. The experimental results show very good quality, with rectified images having a maximum vertical disparity of two pixels, thereby showing that stereo image rectification can be efficiently achieved in an low-resource FPGA (with 64 KB for program instructions and data).