A first order adaptive learning rate algorithm for backpropagation networks

A simple method for determining the learning rate parameter of the backpropagation algorithm is described and analyzed. The learning rate parameter is determined at each step of the iteration by attempting to find a double root of the quadratic cost function. This is opposed to the traditional approach of viewing learning as an optimization problem. It is shown that this method of determining the learning rate parameter leads to accelerated convergence for several benchmark cases