A flexible multimodal object tracking system

In this paper we present a flexible multimodal object tracking system. It is based on a particle filter, which combines the outputs of different measurement methods (also called modes or cues) in a flexible manner. The modes for locating the desired object can be selected depending on the specific object, which is to be tracked, and the environment of the object. By combining multiple modes we aim to add the strengths while at the same time to overcome the specific disadvantages of the different modes. Thus the robustness of the tracking system will be increased and a successful tracking will be possible in critical situations where a system using only a single mode would fail. We have used this approach for tracking persons in different environments. The measurement modes, which we have implemented for this purpose, are a pseudo 2-dimensional hidden Markov model (P2DHMM), a color based skin finder, and a motion detector. We describe the theory and the architecture of this tracking system and finally depict some exemplary results.