Online trajectory following with position based force/vision control

Robot control in uncertain environments can greatly benefit from sensor based control. Visual sensing allows the robot to examine its surroundings and adapt to the environment. Force offers a complementary sensory modality allowing accurate measurements of local object shape when a tooltip is in contact with the object. In multimodal sensor fusion several sensors measuring different modalities are combined together to give more accurate estimate of the environment. We present a method which fuses force and vision in an extended Kalman filter (EKF). A hybrid force controller is then set up to follow a trajectory based on the estimate from the EKF. The estimate allows a simple proportional force control to track a continuous trajectory reliably, where an unfiltered visual measurement becomes unstable. Experiments verify that the method can increase the stability of control considerably.