Robust classification system with reliability prediction for semi-automatic traffic-sign inventory systems

Inventories of traffic signs are acquired from street-level images in a semi-automated fashion, employing object detection and classification techniques. This is a challenging task, as signs are captured from different viewpoints and under various weather conditions. Furthermore, many similar signs exist, only differing in minor details, and moreover, sign-like objects occur frequently. Consequently, current state-of-the-art systems are unable to reach the required quality level, implying the need for manual corrections. This involves checking all classification results to correct the small minority of misclassifications. This paper presents a classification approach aiming at both high recognition scores and predicting the reliability of the classification output, enabling selective manual intervention. Two reliability prediction methods are compared, analyzing either the classifier scores, or matching the input samples with predefined templates. Large-scale experiments performed for three sign classes, each containing numerous sign types, show that over 80% of the correctly classified results can be marked as reliable, while not marking any misclassifications as reliable. Hence, our research shows that a reliable prediction is possible and that manual invention can be concentrated to the about 25% remaining samples only. Overall, 92.7% of the 8, 159 signs are classified correctly.