Computationally efficient implementation of video rectification in an FPGA for stereo vision applications

In order to obtain depth perception in computer vision, it is needed to process pairs of stereo images. This process is computationally challenging to be carried out in real-time, because it requires the search for matches between objects in both images. Such process is significantly simplified if the images are rectified. Stereo image rectification involves a matrix transformation which when done in software will not produce real-time results although it is very demanding. Therefore, the video streaming and matrix transformation are not usually implemented in the same system. Our product is a stereo camera pair which produces a rectified real time image output with a resolution of 320×240 at a frame rate of 15FPS and delivers them via a 100-Ethernet interface. We use a Spartan 3E FPGA for real-time processing within which we implement an image rectification algorithm.