Assisting system of visually impaired in touch panel operation using stereo camera

In this paper, we propose an assisting system of touch panel operation for people with visual disability. In the system, a user specifies the target button on the panel by verbal input. The system detects the button and user´s fingertip by analyzing images obtained through stereo camera. Navigation is made by indicating the direction of the fingertip on the panel through headphones with sound. To construct an efficient navigation method, comparisons were made experimentally concerning to indication of the finger motion direction, choice of navigation sound, and indication of the distance. The effectiveness of the proposed method was verified through experiments.