Hybrid physics simulation of multi-fingered hands for dexterous in-hand manipulation

Dextrous object manipulation with multi-fingered robot hands remains one of the key challenges of service robotics. So far, most theoretical approaches and simulators have concentrated on the search for and evaluation of static stable grasps, but with neither a model of the full hand-arm system nor the system dynamics. GraspIt! is probably the best-known simulator of this kind. In this work we present a simulator that uses the JBullet physics engine to realistically model grasps with multi-fingered hands. It supports manipulation tasks based on a complete arm and hand system, with full calculation of hand and object dynamics. A hybrid dynamics and kinematics approach avoids the oscillations introduced by the different size scale of the arm and hand, so that force-closure grasps are possible in addition to form-closure grasps. The software includes detailed models of our 24-DOF Shadow Dextrous hand and the 6-DOF Mitsubishi PA-10 robot arm. A real-time interface allows us to prepare or to replay and analyze grasp experiments performed on our real robots.