Haptic and visual perception in in-hand manipulation system

Robotic in-hand manipulation is very important for service robots as it is one of the key skills in housework. Although plenty of research has been carried out, it is still far way from real applications. One of the issues lies in uncertainty of interaction states. In this paper we research robot-object interactions with a concept called `haptic exploration´. With this concept, a robot hand tries to push an in-hand object slightly to explore the interaction state. In this process, both haptic and visual feedbacks are collected to estimate the interaction state. We firstly review spring based models proposed in our previous works to explain the mechanism behind the haptic exploration. In addition, in order to verify the feasibility of the proposed methods, two experiments are conducted. In the first experiment, the repeatability of push actions is verified. Furthermore, in the second experiment, both haptic and visual rewards are adopted to evaluate the pushes for a best in-hand manipulation action after the haptic exploration.