Information-theoretic multi-layered supervised self-organizing maps for improved prediction performance and explicit internal representation

In this paper, we propose a new information-theoretic method to train multi-layered neural networks. The method is composed of unsupervised and supervised phase. In the unsupervised phase, the information-theoretic SOM is used to produce knowledge or SOM knowledge in terms of connection weights. In the supervised phase, connection weights obtained in the unsupervised phase are given as the initial connection weights. We applied the method to the segmentation data in machine learning database. The information-theoretic SOM produced connection weights with explicit class boundaries even for the higher layers. By these connection weights, networks reached their lower level of classification errors very rapidly. In addition, the classification error was lower by the choice of the appropriate number of layers.