Title :
Acceleration of Levenberg-Marquardt training of neural networks with variable decay rate
Author :
Chen, Tai-cong ; Han, Da-jian ; Au, Francis T K ; Tham, L.G.
Author_Institution :
Dept. of Civil Eng., South China Univ. of Technol., Guangzhou, China
Abstract :
In the application of the standard Levenberg-Marquardt training process of neural networks, error oscillations are frequently observed and they usually aggravate on approaching the required accuracy. In this paper, a modified Levenberg-Marquardt method based on variable decay rate in each iteration is proposed in order to reduce such error oscillations. Through a certain variation of the decay rate, the time required for training of neural networks is cut down to less than half of that required in the standard Levenberg-Marquardt method. Several numerical examples are given to show the effectiveness of the proposed method.
Keywords :
backpropagation; computational complexity; feedforward neural nets; iterative methods; learning (artificial intelligence); Levenberg-Marquardt method; Levenberg-Marquardt training process; backpropagation; computational complexity; error oscillations; feedforward neural networks; iterative method; variable decay rate; Acceleration; Civil engineering; Computational complexity; Convergence; Jacobian matrices; Least squares methods; Neural networks; Newton method; Optimization methods; Recursive estimation;
Conference_Titel :
Neural Networks, 2003. Proceedings of the International Joint Conference on
Print_ISBN :
0-7803-7898-9
DOI :
10.1109/IJCNN.2003.1223693