Controlling Excessive Information by Surface Information Criterion for Information-Theoretic Self-Organizing Maps

We have so far demonstrated that self-organization maps can be realized by information maximization. However, one problem of information maximization has been pointed out, that is, the excessive information acquisition. In this paper, we try to show that the excessive information can be eliminated by the surface information stopping criterion. In this criterion, we propose two kinds of information, that is, the surface or the deep information. The surface information precedes the deep information, and is an indicator for the future behavior of information increase. By monitoring the surface information, the excessive information can be eliminated. We applied the method to the artificial data and show that a clear feature map can be obtained just by maximizing information. In the second and the third experiment, by the surface information stopping criterion, the maps that exhibit well the characteristics of input patterns can be obtained.