Optimal image processing for multi-target tracking of dismounted targets in an urban environment

Dismounted targets can be tracked in urban environments with video sensors. Real-time systems are unable to process all of the imagery, demanding some method for prioritization of the processing resources. Furthermore, various segmentation algorithms exist within image processing, each algorithm possesses unique capabilities, and each algorithm has an associated computational cost. Additional complexity arises in the prioritization problem when targets become occluded (i.e., a building) and when the targets are intermixed with other dismounted entities. This added complexity leads to the question "which portions of the scene warrant both low cost and high cost processing?" The approach presented in this paper is to apply multi-target tracking techniques in conjunction with an integer programming optimization routine to determine optimal allocation of the video processing resources. This architecture results in feedback from the tracking routine to the image processing function which in turn enhances the ability of the tracker