Face recognition in multi-camera surveillance videos

Recognizing faces in surveillance videos becomes difficult due to the poor quality of the probe data in terms of resolution, noise, blurriness, and varying lighting conditions. In addition, the poses of probe data are usually not frontal view, contrary to the standard format of the gallery data. The discrepancy between the two types of the data makes the existing recognition algorithm less accurate in real-world data. In this paper, we propose a multi-camera video based face recognition framework using a novel image representation called Unified Face Image (UFI), which is synthesized from multiple camera feeds. Within a temporal window the probe frames from different cameras are warped towards a template frontal face and then averaged. The generated UFI is a frontal view of the subject that incorporates information from different cameras. We use SIFT flow as a high level alignment tool to warp the faces. Experimental results show that by using the fused face, the recognition performance is better than the result of any single camera. The proposed framework can be adapted to any multi-camera video based recognition method using any feature descriptors or classifiers.