Information theoretic competitive learning in multi-layered networks

In this paper, we extend our information theoretical competitive learning to multi-layered networks to solve complex problems and to discover salient features that single-layered networks fail to extract. Networks are composed of several competitive layers. In each competitive layer, information is maximized. This successive information maximization enables networks to extract features gradually. We applied the new method to the his data and a phonological data problem. Experimental results confirmed that information can be maximized in multi-layered networks, and the networks can extract features that cannot be detected by single-layered networks.