A New Geometric Approach to the Complexity of Model Selection

Model selection is one of the central problems of machine learning. The goal of model selection is to select from a set of competing explanations the best one that capture the underlying regularities of given observations. The criterion of a good model is generalizability. We must make balance between the goodness of fit and the complexity of the model to obtain good generalization. Most of present methods are consistent in goodness of fit and differ in complexity. But they only focus on the free parameters of the model; hence they cannot describe the intrinsic complexity of the model and they are not invariant under re-parameterization of the model. This paper uses a new geometrical method to study the complexity of the model selection problem. We propose that the integral of the Gauss-Kronecker curvature of the statistical manifold is the natural measurement of the non-linearity of the manifold of the model. This approach provides a clear intuitive understanding of the intrinsic complexity of the model We use an experiment to verify the criterion based on this method