VQ-faces - unsupervised face recognition from image sequences

We propose a new method for unsupervised face recognition - VQ-faces, which operates on a sequential stream of face images and is able to handle both frontal and side-view faces at the same time. The method consists of two parts: in the first part, the VQ-faces are calculated as prototype vectors of local areas in image-space, coding for different face-views (i.e. a "view codebook" is generated), while in the second part the fact that each face-sequence corresponds to a single person (temporal constraint) is used to cluster the different sequences into face categories, using a combinatorial optimization process which maximizes an objective function, reducing the global representation error (distortion) by the VQ-faces as sequences are grouped together. The method was tested on real-world data gathered over a period of several months and including both frontal and side-view faces from 17 different subjects, achieving correct self-organization rate of 85.4%. The applicability of the proposed method is not limited to face recognition - it can be easily applied to other problems involving multi-view object recognition.