Title :
Multimodal speech-gesture interface for handfree painting on a virtual paper using partial recurrent neural networks as gesture recognizer
Author :
Corradini, Andrea ; Cohen, Philip R.
Author_Institution :
Center for Human-Comput. Commun., Oregon Graduate Inst. for Sci. & Technol., Beaverton, OR, USA
fDate :
6/24/1905 12:00:00 AM
Abstract :
We describe a pointing and speech alternative to the current paint programs based on traditional devices like mouse, pen or keyboard. We used a simple magnetic field tracker-based pointing system as input device for a painting system to provide a convenient means for the user to specify paint locations on any virtual paper. The virtual paper itself is determined by the operator as a limited plane surface in the three dimensional space. Drawing occurs with natural human pointing by using the hand to define a line in space, and considering its possible intersection point with this plane. The recognition of pointing gestures occurs by means of a partial recurrent artificial neural network. Gestures along with several vocal commands are utilized to act on the current painting in conformity with a predefined grammar
Keywords :
art; computer graphics; gesture recognition; recurrent neural nets; speech-based user interfaces; gesture recognizer; handfree painting; magnetic field tracker-based pointing system; multimodal speech-gesture interface; paint programs; partial recurrent neural networks; virtual paper; Artificial neural networks; Human computer interaction; Keyboards; Mice; Painting; Paints; Paper technology; Recurrent neural networks; Speech recognition; Tracking;
Conference_Titel :
Neural Networks, 2002. IJCNN '02. Proceedings of the 2002 International Joint Conference on
Conference_Location :
Honolulu, HI
Print_ISBN :
0-7803-7278-6
DOI :
10.1109/IJCNN.2002.1007499