DocumentCode :
387497
Title :
The role of gesture in multimodal referring actions
Author :
Landragin, Frederic
fYear :
2002
fDate :
2002
Firstpage :
173
Lastpage :
178
Abstract :
When deictic gestures are produced on a touch screen, they can take forms which can lead to several sorts of ambiguities. Considering that the resolution of a multimodal reference requires the identification of the referents and of the context ("reference domain") from which these referents are extracted, we focus on the linguistic, gestural, and visual clues that a dialogue system may exploit to comprehend the referring intention. We explore the links between words, gestures and perceptual groups, doing so in terms of the clues that delimit the reference domain. We also show the importance of taking the domain into account for dialogue management, particularly for the comprehension of further utterances, when they seem to implicitly use a pre-existing restriction to a subset of objects. We propose a strategy of multimodal reference resolution based on this notion of reference domain, and we illustrate its efficiency with prototypic examples built from a study of significant referring situations extracted from a corpus. We also present the future directions of our works, concerning some linguistic and task aspects that are not integrated here.
Keywords :
gesture recognition; human factors; interactive systems; multimedia systems; touch sensitive screens; deictic gestures; dialogue management; dialogue system; gestural clues; linguistic clues; multimodal reference; multimodal reference resolution; multimodal referring actions; perceptual groups; reference domain; referring intention; touch screen; visual clues; Avatars; Computer Society; Context modeling; Cultural differences; Humans; Laboratories; Layout; Natural language processing; Prototypes; Speech processing;
fLanguage :
English
Publisher :
ieee
Conference_Titel :
Multimodal Interfaces, 2002. Proceedings. Fourth IEEE International Conference on
Print_ISBN :
0-7695-1834-6
Type :
conf
DOI :
10.1109/ICMI.2002.1166988
Filename :
1166988
Link To Document :
بازگشت