Augmenting SIFT with 3D Joint Differential Invariants for multimodal, hybrid face recognition

Hybrid face recognition methods combine holistic and feature based approaches with the aim of reaching a high level of efficiency and robustness. In this paper we propose a fully automatic algorithm for multimodal data consisting of 2D face images and their corresponding 3D scans. The algorithm is based on the extraction of simple image features using the Scale Invariant Feature Transform and the validation of the corresponding 3D key-points on the scans by means of Joint Differential Invariants based on local and global shape information. The 3D validation process goes through an optimisation procedure: first the invariants are generated from the 3D key-points, then a search for neighbour points that minimise the invariants distance is performed. The total number of matching invariants can be used as a measure of similarity between two faces. The efficacy of the method has been tested on the FRGCv2 database: in particular, experimental results demonstrate the significant contribution of 3D invariants in all cases characterised by a limited number of stable image features.