Square root learning in batch mode BP for classification problems

An algorithm is proposed to increase the learning speed of the standard batch mode BP algorithm for a multilayer perceptron in pattern classification problems. In many problems, the standard batch mode BP suffers from an initial slow learning period. The purpose of the proposed algorithm is to analyze the initial slow learning and to make neural networks converge fast to a local minimum. The key ideas of the proposed algorithm are to use a normalized objective function, to normalize the gradient of the batch mode BP, and to change the learning rate based on the square root of the current gradient norm. The momentum parameter for each weight is also changed according to the normalized gradient. Simulation results demonstrate that the proposed algorithm shortens the initial slow learning period of the batch mode BP and gives a better performance than the online mode BP