DocumentCode :
857774
Title :
A real-time interactive nonverbal communication system through semantic feature extraction as an interlingua
Author :
Hou, Jin ; Aoki, Yoshinao
Author_Institution :
Graduate Sch. of Eng., Hokkaido Univ., Sapporo, Japan
Volume :
34
Issue :
1
fYear :
2004
Firstpage :
148
Lastpage :
154
Abstract :
There has been a growing interest in the use of networked virtual environment (NVE) technology to implement telepresence that allows participants to interact with each other in shared cyberspace. In addition, nonverbal language has attracted increased attention because of its association with more natural human communication, and especially sign languages play an important role for the hearing impaired. This paper proposes a novel real-time nonverbal communication system by introducing an artificial intelligence method into the NVE. We extract semantic information as an interlingua from the input text through natural language processing, and then transmit this semantic feature extraction (SFE) to the three-dimensional (3-D) articulated humanoid models prepared for each client in remote locations. Once the SFE is received, the virtual human is animated by the synthesized SFE. Experiments with Japanese and Chinese sign languages show this system makes the real-time animation of avatars available for the participants when chatting with each other. The communication is more natural since it is not just based on text or predefined gesture icons. This proposed system is suitable for sign language distance training as well.
Keywords :
client-server systems; computer animation; distance learning; gesture recognition; real-time systems; virtual reality; 3D articulated humanoid models; animation; artificial intelligence; avatars; interlingua; natural language processing; networked virtual environment technology; nonverbal language; real-time interactive nonverbal communication systems; semantic feature extraction; sign language distance training; sign languages; telepresence; virtual human; Animation; Artificial intelligence; Auditory system; Data mining; Feature extraction; Handicapped aids; Humans; Natural language processing; Real time systems; Virtual environment;
fLanguage :
English
Journal_Title :
Systems, Man and Cybernetics, Part A: Systems and Humans, IEEE Transactions on
Publisher :
ieee
ISSN :
1083-4427
Type :
jour
DOI :
10.1109/TSMCA.2003.818461
Filename :
1259443
Link To Document :
بازگشت