Talking about 3D scenes: integration of image and speech understanding in a hybrid distributed system

Author

Socher, Gudrun ; Sagerer, Gerhard ; Kummert, Franz ; Fuhr, Thomas

Author_Institution

Bielefeld Univ., Germany

Volume

1

fYear

1996

fDate

16-19 Sep 1996

Firstpage

809

Abstract

We present a hybrid system that integrates speech and image understanding. Given spoken references, it is able to identify objects of a 3D scene perceived via a stereo camera. Central to our approach is the extraction of qualitative object features and spatial scene properties from acoustic and visual data. The interaction of the understanding processes is performed using a procedural semantic network that interfaces with signal recognition and reconstruction modules, thus integrating semantic, neural and Bayesian networks and Hidden Markov Models

Keywords

Bayes methods; computer vision; feature extraction; hidden Markov models; neural nets; semantic networks; speech recognition; visual databases; 3D scenes; Bayesian networks; Hidden Markov Models; hybrid distributed system; image understanding; neural nets; procedural semantic network; qualitative object features extraction; semantic nets; spatial scene properties; speech understanding; spoken references; stereo camera; Cameras; Cognitive science; Data mining; Humans; Image reconstruction; Knowledge representation; Layout; Signal processing; Speech; Stereo vision;

fLanguage

English

Publisher

ieee

Conference_Titel

Image Processing, 1996. Proceedings., International Conference on

Conference_Location

Lausanne

Print_ISBN

0-7803-3259-8

Type

conf

DOI

10.1109/ICIP.1996.561028

Filename

561028