Tracking human speech events using a particle filter

A method of detecting and tracking moving human speech events by integrating audio and video signals using a particle filter is proposed and evaluated. Using the particle filter, location, on/off status, and human/non-human determination can be estimated simultaneously for multiple sound sources. Experiments demonstrate that the proposed method performs well for the data obtained in an ordinary meeting room using a microphone array and a monocular camera.