Implementation of auto-rectification and depth estimation of stereo video in a real-time smart camera system

In this paper, we present a real-time, low power, and wireless embedded stereo vision system. The system consists of WiCa board (wireless camera board) and the methods developed using a smart camera processor, IC3D. IC3D is an SIMD video-analysis processor that has 320 processor elements. The proposed auto-rectification method is suitable for a parallel stereo system like WiCa. It is based on the matching of a planar background. After rectification, the two images of the background are coincident with each other, i.e., both the vertical and horizontal disparity of the background plane is removed. Then a dense matching method is implemented to achieve the depth map of the foreground object. Left-to-right checking and reliability checking is applied to reduce the error of the estimated depth. The system runs at 30 fps and handles disparity up to 37 pixels in CIF (320times240 pixels) mode.