Acquiring 3D motion trajectories of large numbers of swarming animals

Social behavior of animal group, such as insect swarm, bird flock, fish school, has captivated strong interest of scientists in many fields for years. Acquiring 3D motion trajectory of each individual in a swarm is vital for quantitative study of such behavior, yet this task is challenging due to large numbers of individuals, similar visual feature and frequent occlusions. In this paper, we present a novel approach which provides global optimal results for this task by formulating it as three linear assignment problems (LAP). The first LAP obtains the 2D tracks of particles in video sequences via spatially global assignment; the second one utilizes maximum epipolar co-motion length (MECL) to effectively eliminate matching ambiguities; the last one links the track segments into complete 3D trajectories via spatial-temporal global assignment. The proposed matching cost MECL encodes the global motion information during the whole track and is able to handle the association errors resulting from the first LAP. Our method is computationally efficient and works in near real time on a PC. Experiment results on simulated particle swarms validated the accuracy and efficiency of the proposed method. As real-world case, we successfully acquired 3D trajectories of Drosophila melanogaster (fruit fly) swarm comprising hundreds of individuals, which to our best knowledge is the first such achievement.