A low-latency 60 Hz stereo vision system for real-time visual control

The author presents an image acquisition and processing system designed for real-time six-dimensional object tracking with the precise temporal semantics and low latency required for use within the feedback loops of robotic applications. To achieve real-time performance, the system must extract from the image only the information needed to accomplish the task. Successful task performance depends on an integrated system. rather than on a collection of components. The hardware design takes the form of two large complementary VME-compatible cards: TRIAX, capable of simultaneously acquiring and processing two high-resolution video streams, and JIFFE, capable of 20 MFLOPS for the three-dimensional calculations. Algorithms are described that enable the system to track a rectangular solid, intended to track objects held by a robot hand. The algorithms hinge on scanning only small line segments of the image, followed by techniques for reconstructing object location from sparse edge points. The system tracks a cardboard box