Tongue-Movement Communication and Control Concept for Hands-Free Human–Machine Interfaces

A new communication and control concept using tongue movements is introduced to generate, detect, and classify signals that can be used in novel hands-free human-machine interface applications such as communicating with a computer and controlling devices. The signals that are caused by tongue movements are the changes in the airflow pressure that occur in the ear canal. The goal is to demonstrate that the ear pressure signals that are acquired using a microphone that is inserted into the ear canal, due to specific tongue movements, are distinct and that the signals can be detected and classified very accurately. The strategy that is developed for demonstrating the concept includes energy-based signal detection and segmentation to extract ear pressure signals due to tongue movements, signal normalization to decrease the trial-to-trial variations in the signals, and pairwise cross-correlation signal averaging to obtain accurate estimates from ensembles of pressure signals. A new decision fusion classification algorithm is formulated to assign the pressure signals to their respective tongue-movement classes. The complete strategy of signal detection and segmentation, estimation, and classification is tested on four tongue movements of eight subjects. Through extensive experiments, it is demonstrated that the ear pressure signals due to the tongue movements are distinct and that the four pressure signals can be classified with an accuracy of more than 97% averaged across the eight subjects using the decision fusion classification algorithm. Thus, it is concluded that, through the unique concept that is introduced in this paper, human-computer interfaces that use tongue movements can be designed for hands-free communication and control applications.