Using Modality Replacement to Facilitate Communication between Visually and Hearing-Impaired People

Using sign language, speech, and haptics as communication modalities, a virtual treasure-hunting game serves as an entertainment and educational tool for visually and hearing impaired users. In recent years, there has been an increasing interest in human-computer interaction (HCI) for multimodal interfaces. Since Sutherland´s SketchPad in 1961 and Xerox´s Alto in 1973, computer users have long been acquainted with technol ogies other than the traditional keyboard for interacting with a system. Recently, with the desire for increased productivity, seamless in teraction, immersion, and e-inclusion of people with disabilities, along with progress in fields such as multimedia, multimodal signal analy sis, and HCI, multimodal interaction has emerged as an active field of research.