Face Recognition With Image Sets Using Locally Grassmannian Discriminant Analysis

We propose an efficient and robust solution, called locally Grassmannian discriminant analysis (LGDA), for face recognition with image set, where each set contains images belonging to the same subject and typically covering large variations. In our work, by modeling each image set as a nonlinear manifold, linearity-constrained nearest neighborhood clustering is first presented for expressing a manifold by a collection of local linear models (LLMs), each depicted by a subspace. With a proper kernel function defined by canonical correlation between the subspaces, the obtained LLMs can be projected into low-dimensional LGDA embedding space using a set of locally linear transformations. Different from traditional discriminant analysis approaches, LGDA is for multiclass nonlinear discrimination and it can maximize discriminatory power while simultaneously promoting consistency between the multiple local representations of single class objects. A novel accelerated proximal gradient-based learning algorithm is proposed for finding the optimal set of local linear bases. To measure the similarities between the face image sets, three distance criterions are presented, which integrate the distance between the pairs of low-dimensional Grassmannian points from one of the involved manifolds. Comprehensive experiments on the UCSD/Honda, CMU MoBo, and YouTube Celebrities face data sets show that our method consistently outperforms the state of the art.