On comparing the performance of object recognition systems

We give a methodology for the evaluation and comparison of object recognition systems. The methodology is based on indicators of two kinds: (1) statistical and (2) algorithmic. Statistical indicators measure the significance of the performance difference between different systems and rank the performances when the difference is significant. Of the various statistical indicators, we use the Kruskal-Wallis H test. Algorithmic indicators include the usual space and time complexity measures, and various performance curves in variables of error and test data sample size. We illustrate the methodology to evaluate a number of pattern classifiers