Manipulation and grasping control for a hand-eye robot system using sensory-motor fusion

This paper presents a hybrid motion control on a robot arm and hand system, for grasping and manipulating, by integration of multiple sensors such as stereo camera vision, force sensors and surface electromyography(sEMG), to achieve delicate manipulation. The optimization of multi-fingered robotic dexterous grasping-force can be reformulated as an objective function minimization problem in the presence of constrains caused by form-closure and the balance of external force. A recurrent neural network (RNN) method is introduced to address the force optimization problem in real time when the dexterous hand grasp an object. This RNN based method has a simple structure without using complicated models such that the problem of friction constraint linearization can be avoided. Meanwhile, the primal-dual neural algorithm is used to solve the problem of trajectory planning of manipulators in task space, and the effectiveness has been tested by experimental studies.