An improved learning law for backpropagation networks

An updating law for the adaptive selection of the step size (or the learning rate) is introduced. It is especially suitable for pattern learning, and is based on a vectorial analysis to a modified backpropagation network with updatable nonlinearities of neurons. The application to a simulated data set is included to demonstrate the effectiveness of the proposed approach. Some comparisons of performances of networks, with and without updatable nonlinear elements, as well as between the conventional and the proposed updating law, are presented