Multi-camera tracking by joint calibration, association and fusion

To perform surveillance using multiple cameras, camera calibration, measurement-to-object association, fusion of measurements from multiple cameras are three essential components. While these three issues are usually addressed separately, they actually have mutual effects on each other. For example, calibration requires correctly associated objects and measurements with calibration errors will result in wrong associations. In this paper, we present a novel joint calibration, association and fusion approach for multi-camera tracking. More specifically, the expectation-maximization (EM) algorithm is incorporated with the extended Kalman filter (EKF) to give a simultaneous estimate of object states, calibration and association parameters. The real video data collected from two cameras are used to evaluate the tracking performance of the proposed method. Compared to the conventional methods, which perform calibration, association and fusion separately, it is shown that the proposed method can significantly improve the robustness and the accuracy of multi-object tracking.