Optimal classifiers within a Bayesian framework

In recent years, biomedicine has faced a flood of difficult small-sample phenotype discrimination problems. A host of classification rules have been proposed to discriminate types of pathology, stages of disease and other diagnoses. Typically, these classification rules are heuristic algorithms, with very little understood about their performance. To give a concrete mathematical structure to the problem, recent work has utilized a Bayesian modeling framework to both optimize and analyze error estimator performance. We propose to go a step further, using the same Bayesian framework to also optimize classifier design. This would complete a Bayesian theory of classification, where both the classifier error and our estimate of the error may be optimized and studied probabilistically within the assumed model. Optimal classifiers in a discrete model are provided, and we show that performance surpasses that of the popular discrete histogram classifier.