A motion-information-based vision-tracking system with a stereo camera on mobile robots

This paper presents a vision-tracking system for a mobile robot, using robot motion and stereo vision data. The mobile robot has an actuator module which pans and tilts an integrated stereo camera. The proposed system controls the actuator to maintain the line of sight of the stereo camera towards a stationary target by using the robot motion data. The robot motion data are obtained from a gyroscope and encoders of the mobile robot. The stereo vision data from the camera is used to compensate for errors in the motion measurements, and to prevent a long term error accumulation. This vision-based compensation is used only when the robot stops or moves slowly, because the long vision processing times can cause the loop time to overrun. The proposed system is experimentally evaluated while the robot moves on a trajectory.