Title :
Development and convergence analysis of training algorithms with local learning rate adaptation
Author :
Magoulas, G.D. ; Plagianakos, V.P. ; Vrahati, M.N.
Author_Institution :
Dept. of Inf., Athens Univ., Greece
Abstract :
A new theorem for the development and convergence analysis of supervised training algorithms with an adaptive learning rate for each weight is presented. Based on this theoretical result, a strategy is proposed to automatically adapt the search direction, as well as the step-size length along the resultant search direction. This strategy is applied to some well known local learning algorithms to investigate its effectiveness
Keywords :
convergence; feedforward neural nets; gradient methods; learning (artificial intelligence); search problems; batch learning; feedforward neural nets; global convergence; gradient descent method; local learning rate adaptation; search direction; supervised learning; Algorithm design and analysis; Artificial intelligence; Computer networks; Convergence; Error correction; Feedforward neural networks; Informatics; Mathematics; Neural networks; Supervised learning;
Conference_Titel :
Neural Networks, 2000. IJCNN 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on
Conference_Location :
Como
Print_ISBN :
0-7695-0619-4
DOI :
10.1109/IJCNN.2000.857808
