Force and vision resolvability for assimilating disparate sensory feedback

Force and vision sensors provide complementary information, yet they are fundamentally different sensing modalities. This implies that traditional sensor integration techniques that require common data representations are not appropriate for combining the feedback from these two disparate sensors. In this paper, we introduce the concept of vision and force sensor resolvability as a means of comparing the ability of the two sensing modes to provide useful information during robotic manipulation tasks. By monitoring the resolvability of the two sensing modes with respect to the task, the information provided by the disparate sensors can be seamlessly assimilated during task execution. A nonlinear force/vision servoing algorithm that uses force and vision resolvability to switch manipulator control between sensors is proposed. The advantages of the assimilation technique are demonstrated during contact transitions between a stiff manipulator and a rigid environment, a system configuration that easily becomes unstable when force control alone is used. Experimental results show that robust contact transitions are made by the proposed nonlinear controller while simultaneously satisfying the conflicting task requirements of fast approach velocities, maintaining stability, minimizing impact forces, and suppressing bounce between contact surfaces