Autonomous vision-based manipulation from a rover platform

Current rover designs use on-board manipulators to enhance their capabilities for planetary exploration and in-situ science. In this paper, we describe how these manipulators can be used to perform two types of operations: rock sample acquisition for return to Earth, and instrument placement for in-situ science measurements. We describe the computational architecture, tools, and algorithms that we developed for this task. These algorithms integrate rover odometry, stereo visual tracking, and tactile sensing for each operation. We have successfully demonstrated these operations on a self-contained Mars rover prototype, Rocky 7. We have demonstrated the grasping of a small rock sample from a distance of more than one meter away and placing an instrument on a boulder from a distance of more than five meters away