RKLT: 8 DOF Real-Time Robust Video Tracking Combing Coarse Ransac Features and Accurate Fast Template Registration

The performance of a tracker can be measured by two often conflicting criteria - robustness and accuracy. Recently researchers have focused on improving robustness, using adaptive appearance models. However updating the appearance model can cause drift and lower the accuracy of motion (state) estimation. These trackers generally compute 2 degree of freedom(DOF) image translation of the object, and are suited for applications such as surveillance. In contrast, we are interested in tracking objects using high DOF motion models - especially 8DOF homograph models that allow tracking of precise state information (projective 8D or calibrated 3Dworld translations and 3D rotations of the tracked object). Such precise state is required for visual motion control of e.g. robot arms, hands and UAV. To this end, we propose a novel tracking algorithm that combines KLT [8], RANSAC [21] and Inverse Compositional tracker [7]. First we sample a large patch into a set of small patches and track each one using frame-to-frame 2D KLT trackers. An 8 DOF homograph describing the large patch motion is then estimated from the current locations of these KLT trackers using RANSAC while also discarding lost trackers as outliers. Finally, using the RANSAC 8DOF motion estimate as the initial guess, we perform a few iterations of an IC registration tracker. This refines the patch motion to sub-pixel accuracy and avoids drift by registering to the original template. We perform three sets of experiments - one is the standard synthetic Lena convergence benchmark and two use real image sequences from recent datasets - to show that our tracker compares favourably with the state-of-the-art.