Title :
Training feedforward neural networks with the Dogleg method and BFGS Hessian updates
Author :
Perantonis, S.J. ; Ampazis, N. ; Spirou, S.
Author_Institution :
Inst. of Inf. & Telecommun., Nat. Center for Sci. Res. DEMOKRITOS, Athens, Greece
Abstract :
We introduce an advanced optimization algorithm for training feedforward neural networks. The algorithm combines the Broyden-Fletcher-Goldfarb-Shanno (BFGS) Hessian update formula with a special case of trust region techniques, called the Dogleg method, as an alternative technique to line search methods. Simulations regarding classification and function approximation problems are presented which reveal a clear improvement both in convergence and success rates over standard BFGS implementations
Keywords :
Hessian matrices; convergence; feedforward neural nets; function approximation; learning (artificial intelligence); optimisation; pattern classification; BFGS Hessian updates; Broyden-Fletcher-Goldfarb-Shanno Hessian update formula; Dogleg method; advanced optimization algorithm; classification; success rates; trust region techniques; Convergence; Cost function; Electronic mail; Feedforward neural networks; Function approximation; Globalization; Informatics; Mathematics; Neural networks; Search methods;
Conference_Titel :
Neural Networks, 2000. IJCNN 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on
Conference_Location :
Como
Print_ISBN :
0-7695-0619-4
DOI :
10.1109/IJCNN.2000.857827
