Semi-autonomous reference data generation for perception performance evaluation

In the development phase of perception systems (e.g. for advanced driver assistance systems) general interest is pointing towards the performance of the respective detection and tracking algorithms. One common way to evaluate such systems relies on simulated data which is used as a reference. We present a semi-autonomous method, which allows the extraction of reference data from sensor recordings (including data at least from a camera and a distance measuring sensor device). Furthermore, we show how to combine these reference data with the output from the object detection system and how to derive performance statistics (detection and miss rates) of the system. As the generated reference information can be stored along with the sensor recordings, this method also facilitates the comparison of different software versions or algorithm parameters.