Fusion based approach for thermal and visible face recognition under pose and expresivity variation

Many existing works in face recognition are based solely on visible images. The use of bimodal systems based on visible and thermal images is seldom reported in face recognition, despite its advantage of combining the discriminative power of both modalities, under expressions or pose variations. In this paper, we investigate the combined advantages of thermal and visible face recognition on a Principal Component Analysis (PCA) induced feature space, with PCA applied on each spectrum, on a relatively new thermal/visible face database - OTCBVS, for large pose and expression variations. The recognition is done through k-nearest neighbors classification. Our findings confirm that the recognition results are improved by the aid of thermal images over the classical approaches on visible images alone, when a suitably chosen classifier score fusion is employed. We also propose a validation scheme for deriving the optimal fusion score between the two recognition modalities.