Multiple target tracking using cognitive data association of spatiotemporal prediction and visual similarity

Object tracking is crucial to surveillance systems, which provide target information including position, size, and velocity. This paper presents a data association process combining two primary components of visual features and spatiotemporal prediction. In addition, the change perception and the visual distinguishability are utilized to adaptively combine the two primary components. The proposed spatiotemporal prediction is performed on several consecutive frames in order to cover the irregular motion of targets. The prediction is then filtered with a change perception mask to determine whether the candidate observations have motion or not. In addition, the level of contribution of a visual feature is adjusted by the proposed distinguishability to maintain a reward-penalty balance. The proposed method is applied to various video sequences having small targets and abrupt motions, and the experimental results show consistent tracking performance.