Automatic Registration of Color and Infrared Videos Using Trajectories Obtained from a Multiple Object Tracking Algorithm

The registration of images from multiple types of sensors (particularly infrared sensors and color sensors) is the first step to achieving multi-sensor fusion. This is the subject of this paper. Registration is performed by using the trajectories of the moving objects obtained by a novel multiple object tracking algorithm. The trajectories are matched using a RANSAC-based algorithm. The registration criteria used in this algorithm is based on the number of pixels in intersection after overlapping the two images. Results show registration accuracy close to the ground-truth after about 30 frames. The influence of the tracking method used to obtain the trajectories is also investigated.