A multimodal human computer interface combining head movement, speech and tongue motion for people with severe disabilities

Assistive technologies (ATs) play a crucial role in the lives of individuals with severe disabilities by enabling them to have greater autonomy in performing daily tasks. The Tongue Drive System (TDS) developed at the Georgia Tech Bionics Lab is such an AT, empowering people with severe Spinal Cord Injuries (SCIs) to be more independent. Earlier versions of the TDS have offered tongue motion and speech as means of driving mouse activity and keyboard input. In this paper, we introduce a new multi-modal Tongue Drive System (mTDS), which incorporates head tracking to deliver proportional control of a mouse cursor. The mTDS integrates this new capability while preserving tongue motion and speech from previous versions and offers a richer means of driving computing interfaces, than previously available to individuals with severe disabilities. In experimental trials, 3 able bodied subjects attempted to initiate, dictate and send an email using the mTDS. The mean task completion times of expert, intermediate and novice users with mTDS were 1.48, 4.46 and 6.02 times compared to them using a keyboard and mouse interface. We also observed improved user efficacy with repeated use of the mTDS.