Automated language acquisition in multimodal environment

Author

Nagy, Dániel ; Medl, Attila ; Flanagan, James L.

Author_Institution

CAIP Center, Rutgers Univ., Piscataway, NJ, USA

Volume

2

fYear

2000

fDate

2000

Firstpage

937

Abstract

The article describes a real-time laboratory prototype system that adaptively acquires multimodal language and uses it for natural style human-machine communication. Methods for building information theoretic networks and dialogue control are presented. Example interactions are presented and results are analyzed for two modalities: speech and mouse (pen) pointer. Future work is outlined, with special emphasis on the application of multimodal language learning in information transformation

Keywords

information theory; linguistics; multimedia computing; natural language interfaces; real-time systems; automated language acquisition; dialogue control; information theoretic networks; information transformation; mouse pointer; multimodal environment; multimodal language learning; natural style human-machine communication; real-time laboratory prototype system; speech; Communication system control; Feedback; Humans; Information processing; Laboratories; Mice; Natural languages; Prototypes; Software agents; Speech analysis;

fLanguage

English

Publisher

ieee

Conference_Titel

Multimedia and Expo, 2000. ICME 2000. 2000 IEEE International Conference on

Conference_Location

New York, NY

Print_ISBN

0-7803-6536-4

Type

conf

DOI

10.1109/ICME.2000.871513

Filename

871513