Compliant physical interaction based on external vision-force control and tactile-force combination

This paper presents external vision-force control and force-tactile integration in three different examples of multisensor integration for robotic manipulation and execution of everyday tasks, based on a general framework, presented in [1], that enables sensor-based compliant physical interaction of the robot with the environment. The first experiment is a door opening task where a mobile manipulator has to pull open the handle with a parallel jaw gripper by using vision and force sensors in a novel external vision-force coupling approach [2] where the combination is done at the control level; the second one is another vision-force door opening task, but including a sliding mechanism and a different robot, endowed with a three-fingered hand; finally, the third task is to grasp a book from a bookshelf by means of tactile and force integration. The purpose of this paper is twofold: first, to show how vision and force modalities can be combined at the control level by means of an external force loop. And, second, to show how the sensorbased manipulation framework proposed in [1] can be easily applied to very different physical interaction tasks in the real world, allowing for dependable and versatile manipulation.