Information Geometry Approach to the Model Selection of Neural Networks

Model selection is an efficient method to overcome the over-fitting problem of large-scale neural networks. The crux of model selection is generalization. To obtain good generalization we must make balance between the goodness of fit and the complexity of the model. Most of present methods only focus on the parameters of model, which cannot describe the intrinsic complexity of the model. Information geometry is the application of differential geometry in statistical. We studied on the model selection of neural networks use the information geometry method. We propose that the Gauss-Kronecker curvature of the statistical manifold is the natural measurement of the non-linearity of the manifold. This approach provides a clear intuitive understanding of the model complexity