An Informative Interpretation of Decision Theory: Scalar Performance Measures for Binary Decisions

A previous formulation for the application of information accounting to binary decision theory is extended to permit the quality of the decision to be quantitatively measured by evaluation of the underlying informational support. Both a single exemplar measure of information, separability, and its ensemble average equivalent, separation, are shown to measure the information support for decision quality (i.e., how well-informed is the decision), rather than the information support for decision adjudication (i.e., which hypothesis is the better choice) provided by predecision information measures. When compared to the traditional receiver operating characteristic, these measures present several functional advantages. They are scalar in nature, and may be directly optimized over secondary parameters, as well as being rigorously well posed and universally comparable. They incorporate the effects of all relevant decision components (prior information, observational information, and decision rule) in a unified manner while still being easily related to the predecision information measures of log likelihood ratio and generalized signal-to-noise ratio. They can be applied equally well to individual trials or composite averages, and evaluation does not require knowledge of the underlying truth. Compared to false alarm-oriented methods for assessing decision performance, their construction reduces sensitivity to tail effects in the underlying distributions.