Measuring the performance of track-to-track fusion systems

An effort is ongoing to develop a multi-source integration (MSI) system for the E-2C Hawkeye aircraft. The objective of this system development is to merge track information from a variety of dissimilar sources and present to the operator a useful and uncluttered real-time picture of the battlespace to support the current suite of missions being performed by the aircrew onboard the E-2C. In other words, the operator\´s display should have "one track for one piece of metal" and this process should be automatic and accurate. The inputs to the MSI system are tracks from several different sources that arrive at different times and with varying and sometimes unknown accuracies. Thus, the MSI system is a track-to-track fusion system. In order to evaluate the effectiveness of the MSI system in the E-2C operating environment we needed to develop a set of scenarios to run the MSI system against and to define a set of metrics that will quantify the performance of the MSI system with these scenarios. The metrics assume ground truth is known. This work focuses on the development of the metrics that will eventually be used to evaluate the performance of the MSI system. One of the key and unique aspects of these metrics is that they deal with the fact that the track data given to the MSI system is often incorrect or impure. Ideally, each track should be updated with reports from exactly the same platform each time. But since the E-2C operates in environments with large numbers of platforms, it is likely that many of the tracks contain updates from more than one platform. These types of tracks are called impure source tracks. In defining our metrics, we needed to generalize them so that they could deal with both pure and impure source tracks. This work discusses what metrics were defined, how they are computed, and how the metrics were extended to deal with impure tracks.