Uncertainty in the Output of Artificial Neural Networks

The goal for artificial neural networks (ANNs) in two-class classification problems is to predict the class membership accurately. Performance evaluation of ANNs focuses usually on the collective accuracy over a large number of cases in the prediction of the class membership, often measured by receiver operating characteristic (ROC) curve and area under the ROC curve (AUC). We show that with finite number of training cases, the output value of the ANN is a statistical random variable that exhibits uncertainty. We show that this uncertainty in the ANN output can be studied by training multiple ANNs of identical structure on a single set of training cases but with different random initialization, thereby causing the ANNs to arrive at not-necessarily-identical weight values at the conclusion of satisfactory training. We found that this variability in the ANN output is small but not negligible and that it can be important in CAD applications in which the ANN output is to be interpreted by a human observer rather than to be compared with a fixed threshold value in fully automated machine classification.