Multimodal Integration and Learning in Cognitive Technical Systems

Recently, new applications of intelligent systems like service robots and cognitive automobiles have been emerging. The success of these new applications will mainly lie in their ability to incrementally learn through multimodal interaction with their environment and with humans. Advances have been made both in interacting with the environment through the development of elementary sensorimotor skills as well as in human-machine interaction via improved social skills. In this talk, I will first give an overview of the hierarchy of robot skills and the important human-machine interaction modalities. Then I will introduce the incremental-learning approaches to developing self-valued, multisensor-based robot skills. Interactive assembly and service experiments in which a robot is instructed using natural language, gesture and gaze information will also be discussed. Finally, I will summarize the future research topics such as context-awareness, ambient sensing, autonomous credit-assignment, cognitive architecture and cross-modal integration.