Improvement of human hand motion observation by exploiting contact force measurements

The aim of this paper is to present a novel method to improve the observation of the human hand motion, exploiting the measurements of fingertip contact forces. The core idea of the proposed algorithm is to compare the fingertip contact information, obtained by commercial tactile sensors, with the contact information computed in a virtual environment, that reproduces the real environment in which the observation is carried out. In case the estimation of the joint angles and the relative pose between the hand and the object are accurate, the contact information in the virtual and in the real environment are consistent. On the other hand, when the two sources of information are not consistent, a correction of the hand posture is applied. The algorithm has been designed to work on-line. In general, this feature is particulary important for Programming by Demonstration (PbD) applications, since it allows the trainer to actively adapt the demonstration to measurement noise and model errors. The effectiveness of the proposed method has been tested in three different tasks: grasping a cup, unscrewing a bottle, grasping a plate.