Model selection for a medical diagnostic decision support system: a breast cancer detection case

There are a number of different quantitative models that can be used in a medical diagnostic decision support system (MDSS) including parametric methods (linear discriminant analysis or logistic regression), non-parametric models (K nearest neighbor, or kernel density) and several neural network models. The complexity of the diagnostic task is thought to be one of the prime determinants of model selection. Unfortunately, there is no theory available to guide model selection. Practitioners are left to either choose a favorite model or to test a small subset using cross validation methods. This paper illustrates the use of a self-organizing map (SOM) to guide model selection for a breast cancer MDSS. The topological ordering properties of the SOM are used to define targets for an ideal accuracy level similar to a Bayes optimal level. These targets can then be used in model selection, variable reduction, parameter determination, and to assess the adequacy of the clinical measurement system. These ideas are applied to a successful model selection for a real-world breast cancer database. Diagnostic accuracy results are reported for individual models, for ensembles of neural networks, and for stacked predictors.