Parallel tangent methods with variable stepsize

The most widely used algorithm for training multilayer feedforward neural networks is backpropagation. Back-propagation is an iterative gradient descent algorithm. Since its appearance, various methods which modify the conventional BP have been created to improve its efficiency. One such algorithm which uses an adaptive learning rate is backpropagation with variable stepsize, is proposed. Parallel tangent methods are used in global optimization to modify and improve the simple gradient descent algorithm by using from time to time the difference between the current point and the point before two steps as a search direction, instead of using the gradient. In this study, we investigate the combination of the BPVS method with the parallel tangent approach for neural network training. We perform experimental results on well-known test problems to evaluate the efficiency of the method.