Using eye contact and contextual speech recognition for hands-free surgical charting

In this paper we discuss ongoing research into applications for multimodal Attentive User Interfaces in hands-free charting during surgical procedures. Although speech recognition has matured enough that it can now be used for some software and hardware control, speech recognition solutions still have trouble filtering ldquocommand speechrdquo from ldquoambient speech.rdquo Our research builds on previous research that couples eye contact sensing with speech recognition to gauge intent. Users enable a voice activation system used for surgical time charting by fixing their gaze on a small camera before speaking command words.