Rejection Schemes in Multi-class Classification -- Application to Handwritten Character Recognition

The recognition of handwritten characters is an almost solved problem thanks to efficient machine learning techniques. However, the evaluation and the choice of thresholds to meet a certain level of performance remains a challenge. In this paper, we compare different rejection techniques to determine if a character has been successfully detected or not. Whereas the evaluation of binary classifiers through ROC curves and cost curves has been largely exploited in the literature, multi-class problems can involve different issues like the computational cost or the choice of having an adaptive threshold for each class. The proposed methods for rejection criteria include the maximization of the distance to the optimal performance in the ROC space. We show that optimizing this criterion is a suitable approach on two databases: Lampung handwritten characters, and Arabic handwritten digits. The results support the conclusion that in database where the accuracy is high, pair wise ROC curves analysis in multi-class problems can lead to a finer evaluation of the performance and thresholds definition.