Grasp recognition using a 3D articulated model and infrared images

A technique to recognize the shape of a grasping hand during manipulation tasks is proposed; which utilizes 3D articulated hand model and a reconstructed 3D volume from infrared cameras. Vision-based recognition of a grasping hand is a tough problem, because a hand may be partially occluded by a grasped object and the ratio of occlusion changes along the progress of the task. To recognize the shape in a single time frame, a robust recognition method of an articulated object is proposed. In this method, 3D volumetric representation of a hand is reconstructed from multiple silhouette images and 3D articulated object model is fitted to be reconstructed data to estimate the pose and the joint angles. To deal with large occlusion, a technique to simultaneously estimate time series reconstructed volumes with the above method is proposed, which can automatically suppress the effect form badly reconstructed volumes. The proposed techniques are verified in simulation as well as in a real world.