Person Tracking with Audio-Visual Cues Using the Iterative Decoding Framework

Tracking humans in an indoor environment is an essential part of surveillance systems. Vision based and microphone array based trackers have been extensively researched in the past. Audio-visual tracking frameworks have also been developed. In this paper we consider human tracking to be a specific instance of a more general problem of information fusion in multimodal systems. Dynamic Bayesian networks have been the modeling technique of choice to build such information fusion schemes. The complexity and non-Gaussianity of distributions of the dynamic Bayesian networks for such multimodal systems have led to the use of particle filters as an approximate inference technique. In this paper we present an alternative approach to the information fusion problem. The iterative decoding algorithm is based on the theory of turbo codes and factor graphs used in communication systems. We modify and adapt the iterative decoding algorithm to do probabilistic inference for the problem of tracking humans in an indoor space, using multiple cameras and microphone arrays.