Recognition of Hand Contact State with an Omnidirectional Camera and Laser Modules

In this paper, we propose a human-machine interface to recognize the user´s finger motion in order to perform object manipulation with a multi-fingered robot hand. The information needed for object manipulation includes the desired contact positions between robot fingers and the manipulated object as well as whether or not each finger should touch the object. Our system obtains the above information by using an omni directional camera and laser modules, while the user holds the camera with fingers by regarding the camera as a virtual manipulated object. After the features are extracted from the image which contains laser light reflecting on the user´s fingers, the features are used for a support vector machine to recognize the user´s contact state. The experimental results show that our system recognizes the contact state in real-time at more than 92 percent accuracy rate.