Title :
Stability of steepest descent with momentum for quadratic functions
Author :
Torii, Manabu ; Hagan, Martin T.
Author_Institution :
Delaware Univ., Newark, DE, USA
fDate :
5/1/2002 12:00:00 AM
Abstract :
This paper analyzes the effect of momentum on steepest descent training for quadratic performance functions. We demonstrate that there always exists a momentum coefficient that will stabilize the steepest descent algorithm, regardless of the value of the learning rate. We also demonstrate how the value of the momentum coefficient changes the convergence properties of the algorithm
Keywords :
backpropagation; eigenvalues and eigenfunctions; neural nets; quadratic programming; stability; backpropagation learning algorithm; learning rate; momentum coefficient; quadratic functions momentum; quadratic performance functions; steepest descent stability; steepest descent training; Algorithm design and analysis; Backpropagation algorithms; Convergence; Cost function; Eigenvalues and eigenfunctions; Equations; Helium; Performance analysis; Stability; Symmetric matrices;
Journal_Title :
Neural Networks, IEEE Transactions on
DOI :
10.1109/TNN.2002.1000143
