Coupled Hidden Markov Models for Robust EO/IR Target Tracking

Augmenting electro-optical (EO) based target tracking systems with infrared (IR) modality has been shown to be effective in increasing the accuracy rate of the tracking system. A key issue in designing such a multimodal tracking system is how to combine information observed from different sensor types in a systematic way to obtain desirable performance. In this paper, we present an investigation into integrating EO and IR sensors within hidden Markov model (HMM) based frameworks. We propose to use a coupled hidden Markov model (CHMM) to improve upon the existing fusion schemes. Another contribution is that we propose to use a robust t-distribution based subspace representation in the CHMM to model appearance changes of the target. Numerical experiments demonstrate that the proposed CHMM tracking system has improved performance over other integration schemes for situations where the target object is corrupted by noise or occlusion.