Neural network information criterion for the optimal number of hidden units

This paper presents a statistical approach to solve the problem of model selection, or determine the number of hidden units for artificial neural networks. The authors´ approach analyzes the relation between the learning error which is a measure of the convergence and the general generalization error which is a measure of the quality of approximation by a statistical discrepancy and a discrepancy accompanied by the learning algorithm. The author makes clear the relation between the learning error and the generalization error by analyzing the two discrepancies statistically. Moreover, the author derives a new information criterion based on a given learning set by this relation. The author calls this information criterion a “neural network information criterion: NNIC”. The author can construct the optimal architecture of multi-layered neural networks for given learning examples by using this criterion. This paper shows that this criterion is effective by a simple simulation which compares NNIC with NIC. Finally, the author points out that NNIC is the generalized form of some information criteria