Title :
Semi-Automatically Generated High-Level Fusion for Multimodal User Interfaces
Author :
Ertl, Dominik ; Kavaldjian, Sevan ; Kaindl, Hermann ; Falb, Jürgen
Author_Institution :
Inst. of Comput. Technol., Vienna Univ. of Technol., Vienna, Austria
Abstract :
Reliable high-level fusion of several input modalities is hard to achieve, and(semi-)automatically generating it is even more difficult. However, it is important to address in order to broaden the scope of providing user interfaces semi-automatically.Our approach starts from a high-level discourse model created by a human interaction designer. It is modality independent, so an annotated discourse is semiautomatically generated, which influences the fusion mechanism. Our high-level fusion checks hypotheses from the various input modalities by use of finite state machines. These are modality independent, and they are automatically generated from the given discourse model. Taking all this together, our approach provides semi-automatic generation of high-level fusion. It currently supports input modalities graphical user interface (simple) speech, a few hand gestures, and a bar code reader.
Keywords :
finite state machines; formal logic; user interfaces; annotated discourse; bar code reader; finite state machines; graphical user interface; hand gestures; high-level discourse model; human interaction designer; input modalities; multimodal user interfaces; semi-automatically generated high-level fusion; speech; Automata; Computer interfaces; Fusion power generation; Graphical user interfaces; Humans; Rendering (computer graphics); Robots; Speech coding; Speech recognition; User interfaces;
Conference_Titel :
System Sciences (HICSS), 2010 43rd Hawaii International Conference on
Conference_Location :
Honolulu, HI
Print_ISBN :
978-1-4244-5509-6
Electronic_ISBN :
1530-1605
DOI :
10.1109/HICSS.2010.335