• DocumentCode
    744677
  • Title

    Stability of steepest descent with momentum for quadratic functions

  • Author

    Torii, Manabu ; Hagan, Martin T.

  • Author_Institution
    Delaware Univ., Newark, DE, USA
  • Volume
    13
  • Issue
    3
  • fYear
    2002
  • fDate
    5/1/2002 12:00:00 AM
  • Firstpage
    752
  • Lastpage
    756
  • Abstract
    This paper analyzes the effect of momentum on steepest descent training for quadratic performance functions. We demonstrate that there always exists a momentum coefficient that will stabilize the steepest descent algorithm, regardless of the value of the learning rate. We also demonstrate how the value of the momentum coefficient changes the convergence properties of the algorithm
  • Keywords
    backpropagation; eigenvalues and eigenfunctions; neural nets; quadratic programming; stability; backpropagation learning algorithm; learning rate; momentum coefficient; quadratic functions momentum; quadratic performance functions; steepest descent stability; steepest descent training; Algorithm design and analysis; Backpropagation algorithms; Convergence; Cost function; Eigenvalues and eigenfunctions; Equations; Helium; Performance analysis; Stability; Symmetric matrices;
  • fLanguage
    English
  • Journal_Title
    Neural Networks, IEEE Transactions on
  • Publisher
    ieee
  • ISSN
    1045-9227
  • Type

    jour

  • DOI
    10.1109/TNN.2002.1000143
  • Filename
    1000143
    • Link To Document
    https://search.isc.ac/dl/search/defaultta.aspx?DTC=49&DC=744677