Statistical analysis of multilayer perceptrons performances

The paper is based on a series of studies on the learning capabilities of multilayered perceptrons (MLP). The complexity of these nonlinear systems can be varied, acting for instance on the number of hidden units, but we will be confronted with a choice dilemma, concerning the optimal complexity of the system for a given problem. By the mean of statistical methods, we have found that the effective number of hidden units is smaller than the potential size; some units have a “binary” activation level or a time constant activation. We also prove that weight initialization to small values is recommended and reduce the effective size of the hidden layer