ALADIN: algorithms for Learning and Architecture DetermINation

This paper presents the development of learning algorithms which are capable of selecting and training the simplest feed-forward neural network for a given application. This is achieved by deactivating the redundant hidden units during the training of the network on the basis of a criterion relating to the effect of each hidden unit on the training process. The information inherent in the training set is subsequently distributed over the remaining active hidden units. In addition to the algorithms based on the quadratic error criterion frequently used for the training of neural networks, this paper also presents the development of fast algorithms based on a new generalized criterion which accelerates the training of neural networks. The proposed algorithms are experimentally evaluated and tested