Growing artifical neural networks based on correlation measures, task decomposition and local attention neurons

We propose a learning architecture for growing complex artificial neural networks. The complexity of the growing network is adapted automatically according to the complexity of the task. The algorithm generates a feedforward network bottom up by cyclically inserting cascaded hidden layers. Inputs of a hidden layer unit are locally restricted with respect to the input space by using a new kind of activation function. Contrary to the cascade-correlation learning architecture, we introduce different correlation measures to train the network units featuring different goals. The task decomposition between subnetworks is done by maximizing the anticorrelation between the hidden layer units output and a connection routing algorithm between the hidden layers. These features resembles the TACOMA (task decomposition, correlation measures and local attention neurons) learning architecture. Results are shown for two difficult to solve problems in comparison to those produced by the CASCOR algorithm