Convergence of coherent components of neural networks by positive correlation learning

We propose a training algorithm for multi-layered neural-network (NN) classifiers that improves the generalization ability, the ability to classify previously unseen patterns. A correlation term was added to the error function of the standard back-propagation (BP) learning algorithm, and a new weight update rule was derived. The correlation term was to make hidden node activations positively correlated, and thus we called the algorithm the positive correlation learning (PCL). The PCL was tested with the standard benchmark problems, the breast cancer and diabetes problems, and the results exhibited that it can produce networks with high generalization ability. It has been known that the minimization of the information in hidden node activations, as well as decaying weights, improves the generalization performance. We here show empirically that the PCL not only reduces the information but also decays the weights effectively