MADALINE RULE II: a training algorithm for neural networks

A novel algorithm for training multilayer fully connected feedforward networks of ADALINE neurons has been developed. Such networks cannot be trained by the popular backpropagation algorithm, since the ADALINE processing element uses the nondifferentiable signum function for its nonlinearity. The algorithm is called MRII for MADALINE RULE II. Previously, MRII successfully trained the adaptive ´descrambler´ portion of a neural network system used for translation invariant pattern recognition. Since then, studies of the algorithm´s convergence rates and its ability to produce generalizations have been made. These were conducted by training networks with MRII to emulate fixed networks. The authors present the principles and experimental details of the MRII algorithm. Typical learning curves show the algorithm´s efficient use of training data. Architectures that take advantage of MRII´s quick learning to produce useful generalizations are presented.<>