Towards unlocking web video: Automatic people tracking and clustering

This paper describes a system for automatically extracting meta-information on people from videos on the Web. The system contains multiple modules which automatically track people, including both faces and bodies, and clusters the people into distinct groups. We present new technology and significantly modify existing algorithms for body-detection, shot-detection and grouping, tracking, and track-clustering within our system. The system was designed to work effectivity on Web content, and thus exhibits robust tracking and clustering behavior over a broad spectrum of professional and semi-professional video content. In order to quantify and evaluate our system we created a large ground-truth data-set of people within video. Finally, we provide actual video examples of our algorithm and find that the results are quite strong over a broad range of content.