How to maintain information content in artificial neural networks with coherence adaptation

This paper presents a learning approach called adaptive coherence scheme (CAS) that adaptively reduces information on input patterns in hidden layer(s) of a neural network. The hidden units in a neural network store information continuously during training session. As a result the network becomes extremely familiar with every details of input patterns. This is not desirable in training. Therefore, we attempt to limit this information automatically with a regularization function consisting of activations of hidden units. We proposed standard coherence learning (SCL) where a constant coherence strength was used in order to solve the problem. Here, we attempt to develop a coherence adaptation scheme in order to maintain small amount of information in the network automatically. We have applied the algorithm to the breast cancer classification and Mackey-Glass chaotic time series prediction problems with single and double hidden layered networks. The results show that the network maintains small amount of information with good classification and prediction accuracies.