A new fast multilayer perceptron training procedure based on the Davidon Fletcher Powell algorithm

The Davidon Fletcher Powell (DFP) optimization algorithm usually used for nonlinear least squares is presented and is combined with the standard backpropagation (SBP) algorithm yielding a new fast training multilayer perceptron (MLP) algorithm (DFP/SBP). The new algorithm is tested on several function approximation problems. The number of iterations required by this algorithm to converge is less than 40% of what is required by the SBP algorithm. Also it is less affected by the choice of initial weights and setup parameters. The DFP/SBP algorithm is much more efficient than either of other techniques when the network contains no more than few hundred weights.