Evaluation of Algorithms for Tracking Multiple Objects in Video

As video tracking research matures, the issue of tracker performance evaluation has emerged as a research topic in its own right, as evidenced by a series of workshops devoted solely to this purpose (the workshops on performance evaluation of tracking and surveillance-PETS). However, evaluations such as PETS have been limited to small scenarios with a handful of moving objects. In this paper, we present an evaluation methodology and set of experiments focused on large-scale video tracking problems with hundreds of objects in close proximity. The scale and complexity of this data exposes a number of issues. First, the association of computed tracks to image-truth tracks may have multiple plausible solutions, resulting in a combinatorial grouping problem that must be solved with an approximate solution. Second, computed tracks may be only partially correct, complicating the association problem further and indicating that multiple measures are required to characterize performance. We have created a system that associates computed tracks to manually-generated image-truth tracks, and calculates various measures such as the per-frame probability of detection, false alarm rate, and fragmentation, which is the number of computed tracks associated to a single track. We also normalize fragmentation by track length to reward fewer computed tracks for longer true tracks. The measures were used to compare three tracking methods on an aerial video sequence containing hundreds of objects, long occlusions, and deep shadows.