Multi-modal force/vision sensor fusion in 6-DOF pose tracking

Sensor based robot control allows manipulation in dynamic and uncertain environments. Vision can be used to estimate 6-DOF pose of an object by model-based pose-estimation methods, but the estimate is not accurate in all degrees of freedom. Force offers a complementary sensor modality allowing accurate measurements of local object shape when the tooltip is in contact with the object. As force and vision are fundamentally different sensor modalities, they cannot be fused directly.We present a method which fuses force and visual measurements using positional information of the end-effector. By transforming the position of the tooltip and the camera to a same coordinate frame and modeling the uncertainties of the visual measurement, the sensors can be fused together in an Extended Kalman filter. Experimental results show greatly improved pose estimates when the sensor fusion is used.