A multi-stream bimodal continuous speech recognition system using datasieve based features

Author

Xie, Lei ; Ravyse, Ilse ; Jiang, Dong-mei ; Zhao, Ong-chun ; Sahli, Hichem ; Verhelst, Werver ; Cornelis, Jan

Author_Institution

Dept. of Comput. Sci. & Eng., Northwestern Polytech. Univ., Xi´´an, China

Volume

4

fYear

2003

fDate

2-5 Nov. 2003

Firstpage

227

Abstract

This paper presents an audio visual bimodal continuous speech recognition system. The visual feature extraction of the mouth movements uses the number of granules obtained by applying a datasieve. Multi-stream HMMs are introduced for combining audio and visual modalities using time synchronous audio visual features. Experimental results show that the recognition system provided by this paper is suitable for continuous speech recognition tasks in noisy environments, and the datasieve based visual features outperform the conventional DCT and DWT features.

Keywords

audio-visual systems; feature extraction; hidden Markov models; speech recognition; audio visual speech recognition; bimodal continuous speech recognition; datasieve based features; feature extraction; mouth movement; multistream HMM; time synchronous audio visual features; Acoustic noise; Discrete cosine transforms; Discrete wavelet transforms; Feature extraction; Humans; Information analysis; Mouth; Speech analysis; Speech recognition; Working environment noise;

fLanguage

English

Publisher

ieee

Conference_Titel

Machine Learning and Cybernetics, 2003 International Conference on

Print_ISBN

0-7803-8131-9

Type

conf

DOI

10.1109/ICMLC.2003.1259888

Filename

1259888