Probabilistic Speaker Localization in Noisy Environments by Audio-Visual Integration

In this paper, we have developed not only a probabilistic sound localization system including VAD (voice activity detection) component using three microphones but also a face tracking system using a vision camera. Moreover, we have proposed a way to integrate these systems to compensate the errors in the localization of a speaker and to reject unnecessary speech or noise signals entering from the undesired directions effectively. For the purpose of verifying our system´s performances, we have installed the proposed audition and vision system to the prototype robot, called IROBAA (intelligent robot for active audition), and showed how to integrate an audio-visual system