Towards the integration of spontaneous speech and gesture based on spotting method

Author

Nishimura, Takuichi ; Oka, Ryuichi

Author_Institution

Real World Comput. Partnership, Tsukuba Res. Center, Ibaraki, Japan

fYear

1996

fDate

8-11 Dec 1996

Firstpage

433

Lastpage

437

Abstract

A spotting algorithm to recognize the meanings of human gestures from motion images is proposed. Feature vectors are extracted from temporal edge image. And input image sequences are matched with models by spotting algorithm. The spotting algorithm removes the need for temporal segmentation of gesture duration and introduces frame-wise recognition. In this paper, a new spotting method called nonmonotonic continuous DP is proposed for spotting gestures and their variations such as reverse, partial and stop motions. We showed the effectiveness of the new spotting method

Keywords

edge detection; image recognition; image sequences; motion estimation; user interfaces; frame-wise recognition; gesture duration; human gestures; input image sequences; motion images; nonmonotonic continuous DP; spontaneous gesture; spontaneous speech; spotting algorithm; temporal edge image; Feature extraction; Hidden Markov models; Humans; Image recognition; Image segmentation; Image sequences; Pattern matching; Pattern recognition; Speech recognition; User interfaces;

fLanguage

English

Publisher

ieee

Conference_Titel

Multisensor Fusion and Integration for Intelligent Systems, 1996. IEEE/SICE/RSJ International Conference on

Conference_Location

Washington, DC

Print_ISBN

0-7803-3700-X

Type

conf

DOI

10.1109/MFI.1996.572213

Filename

572213