Title :
Variable Projection Method and Levenberg-Marquardt Algorithm for Neural Network Training
Author :
Kim, Cheol-Taek ; Lee, Ju-Jang ; Kim, Hyejin
Author_Institution :
KAIST, Daejeon
Abstract :
An optimal solution for single hidden layered feedforward neural network (SLFN) is proposed. SLFN can be considered as separable nonlinear least squares problem and variable projection (VP) method gives the approximation of the Jacobian matrix of the problem. The Jacobian calculation of VP-SLFN is suggested with simplified form. Based on this simplification, Levenberg-Marquardt (LM) algorithm for VP-SLFN is suggested and has faster convergence rate than LM algorithm without VP. Two numerical examples show the superiority than extreme learning machine and LM without variable projection
Keywords :
Jacobian matrices; feedforward neural nets; learning (artificial intelligence); least squares approximations; Jacobian matrix; Levenberg-Marquardt algorithm; learning machine; neural network training; separable nonlinear least squares problem; single hidden layered feedforward neural network; variable projection method; Feedforward neural networks; Function approximation; Iterative algorithms; Jacobian matrices; Least squares approximation; Least squares methods; Machine learning; Neural networks; Newton method; Recursive estimation;
Conference_Titel :
IEEE Industrial Electronics, IECON 2006 - 32nd Annual Conference on
Conference_Location :
Paris
Print_ISBN :
1-4244-0390-1
DOI :
10.1109/IECON.2006.347644
