Adding monotonicity to learning algorithms may impair their accuracy

Ordinal (i.e., ordered) classifiers are used to make judgments that we make on a regular basis, both at work and at home. Perhaps surprisingly, there have been no comprehensive studies in the scientific literature comparing the various ordinal classifiers. This paper compares the accuracy of five ordinal and three non-ordinal classifiers on a benchmark of fifteen real-world datasets. The results show that the ordinal classifiers that were tested had no meaningful statistical advantage over the corresponding non-ordinal classifiers. Furthermore, the ordinal classifiers that guaranteed monotonic classifications showed no meaningful statistical advantage over a majority-based classifier. We suggest that the tested ordinal classifiers did not properly utilize the order information in the presence of non-monotonic noise.