A closed-loop system for rapid face retrieval by combining EEG and computer vision

Computer vision (CV) has demonstrated its capabilities of face retrieval in rapidly exploring large image collections, but its performance degrades in uncontrolled condition. Human brain has powerful cognitive ability to recognize faces across various conditions, which could complement computer face retrieval. In this paper, we propose a closed-loop face retrieval system by iterative coupling between EEG-based target image labeling and CV based label propagation. The closed-loop face retrieval system starts with a sample image of the target face sent to a CV module. The CV module ranks all face images in the database based on their similarity to the sample. The top-ranked 160 face images are presented by RSVP paradigm and an EEG interest detector labels the images of the user´s highest interest scores. We propose a new method to remove false targets from EEG labeled images. The user´s interest score of each EEG labeled image is concatenated with the similarity value to the sample by CV as a feature vector. The feature vector is inputted into a classifier and the face images classified as positive ones are sent to the CV module to close the loop. The experiment results show that our method can remove false targets from EEG labeled images effectively, and the performance of our system is 11.86% and 12.11% superior to the two previous systems, respectively, and indicate that human ability can complement computer face recognition in a close loop for a better face retrieval.