Harvesting Web Images for Realistic Facial Expression Recognition

Large amount of labeled training data is required to develop robust and effective facial expression analysis methods. However, obtaining such data is typically a tedious and time-consuming task that is proportional to the size of the database. Due to the rapid advance of Internet and Web technologies, it is now feasible to collect a tremendous number of images with potential label information at a low cost of human effort. Therefore, this paper proposes a framework to collect realistic facial expression images from the web so as to promote further research on robust facial expression recognition. Due to the limitation of current commercial web search engines, a large fraction of returned images is not related to the query keyword. We present a SVM based active learning approach to selecting relevant images from noisy image search results. The resulting database is more diverse with more sample images, compared with other well established facial expression databases CK and JAFFE. Experimental results demonstrate that the generalization of our web based database outperforms those two existing databases. It is anticipated that further research on facial expression recognition or even affective computing will not be restricted to traditional 7 categories only.