A new criteria for comparing neural networks and Bayesian classifier

The classification in high dimension spaces remains the essential problem for pattern recognition because of the limited samples´ size. Indeed, the dimension reduction is often required in the first step. For complexity and simplicity reasons, the linear methods are the most commonly used ones. The neural techniques can achieve a non-linear dimension reduction, which is qualified by the non-parametric methods. These neural techniques are opposed to recent ones that try to find a sub space in which the data are well presented. Thus, the neural approach offers some flexibility and a tangible usefulness as it often presents the technological solution. However, the lack of control over its mathematical formulation explains the instability of its classification results. We propose in this paper a comparative study between the statistical and the neural approaches. The core of this study is based on the bias and variance of the error rate. Experimentation is performed on simulations and handwritten digit images.