Pointing system using fingers on keyboard

Our final goal is to realize a system where the fingers on a keyboard can be used as a mouse. Such a system realizes more smooth operativity because the user needs not to move his/her hand between the keyboard and the mouse. We realized a function to discriminate between ldquopointing moderdquo (the mode to move the cursor according to the position of the fingertip) and ldquocharacter input moderdquo (the mode to input the character using the keyboard) based on handshape recognition using the image captured by a camera. A mouse click could also be realized by detecting the motion of a thumb in the pointing mode. As a result, a pointing system has been realized where both the pointing operation and the mouse click are possible using the motion of the right hand only on the keyboard.