Real-time 3D recognition of manipulated object by robot hand using 3D sensor

In robotic manipulation, visual servoing is an important technique for achieving dexterous and accurate handling of objects. One of the most important problems in this task is occlusion by the robot´s body. In some situations, it is difficult to know the position and orientation of the object during manipulation because it is hidden by the robot itself. If its position and orientation could be estimated by considering this occlusion, the robot would be able to manipulate it by visual servoing more dexterously. In this paper, we propose an algorithm for estimating the position and orientation of the manipulated object during manipulation by using a 3-D sensor. 3-D information of the total environment is observed by the 3-D sensor, and the observed 3-D point-cloud information is classified into the manipulator, the object, and the others based on their 3-D models. The proposed algorithm was verified in an actual robot manipulation system.