Augmented faces

In this paper we present a fully automatic system for face augmentation on mobile devices. A user can point his mobile phone to a person and the system recognizes his or her face. A tracking algorithm overlays information about the identified person on the screen, thereby achieving an augmented reality effect. The tracker is running on the mobile client, while the recognition is running on a server. The database on the server is built by a fully autonomous crawling method, which taps social networks. For this work we collected 300 000 images from Facebook. The social context gained during this social network analysis is also used to improve the face recognition. The complete system runs in real time on a state-of-the-art mobile phone and is fully automatic, from offline crawling up to augmentation on the mobile device. It can be used to display more information about the identified persons or as a user interface for mixed reality application. To the best of our knowledge this is the first work, which covers such a system end-to-end.