Real-time feature tracking and projective invariance as a basis for hand-eye coordination

This article presents a methodology for using stereo visual feedback to perform manipulation tasks. The two major innovations in the approach are: 1) the use of feature-based tracking methods that perform in real-time on standard workstations without specialized hardware; and 2) the use of closed-loop feedback control based on projective invariants to make positioning accuracy independent of hand-eye calibration error. Particular attention is given to the feature tracking component of the system. The feature tracker is a programming environment that supports a variety of low-level detection methods (basic features), and features defined in terms of other features (composite features). Basic and composite features are combined into feature networks. Experimental results from two feature networks are presented. One computes corresponding epipolar lines using eight corresponding features in two images. The second computes a visual trajectory for a visual servoing system