Vision-based detection of finger touch for haptic device using transparent flexible sheet

A haptic device using transparent flexible sheet, which is an improvement of our previous one, presents visual and haptic sensations when a user pushes a virtual soft object. It consists of a transparent flexible sheet and a computer display; the display is placed behind the sheet. The user feels the softness of the object by pushing the sheet, and sees the stereoscopic CG image of the deformed object through the sheet. In the present study we propose a new method of detecting finger touch on the sheet and measuring the indentation of the contact point; the method is specialized for this haptic device. A camera is placed behind the sheet. A checkered pattern is reflected on the backside of the sheet like a two-way mirror. The camera monitors this reflected pattern. When the finger pushes the sheet, the sheet is deformed. Then the reflected pattern near the fingertip, seen from the camera, is distorted. Monitoring this distortion makes it possible to detect finger touch on the sheet. As the indentation of the contact point increases, the reflected pattern is more distorted. Thus measuring the amount of the distortion allows the measurement of the indentation. The proposed method is ascertained by some experiments.