Title :
A modified hidden weight optimization algorithm for feedforward neural networks
Author :
Yu, Changhua ; Manry, Michael T.
Author_Institution :
Dept. of Electr. Eng., Texas Univ., Arlington, TX, USA
Abstract :
The output weight optimization-hidden weight optimization (OWO-HWO) feedforward network training algorithm alternately solves linear equations for output weights and reduces a separate hidden layer error function with respect to hidden layer weights. Here, a new hidden layer error function is proposed which de-emphasizes net function errors that correspond to saturated activation function values. In addition, an adaptive learning rate based on the local shape of the error surface is used in hidden layer training. Faster learning convergence is experimentally verified.
Keywords :
adaptive systems; convergence of numerical methods; error analysis; feedforward neural nets; learning (artificial intelligence); optimisation; OWO-HWO feedforward network training algorithm; adaptive learning rate; error surface; feedforward neural networks; hidden layer error function; hidden layer training; hidden layer weights; learning convergence; linear equations; local shape; modified hidden weight optimization algorithm; output weight optimization-hidden weight optimization; saturated activation function; Convergence; Equations; Error correction; Feedforward neural networks; Feedforward systems; Gradient methods; Joining processes; Neural networks; Shape; Training data;
Conference_Titel :
Signals, Systems and Computers, 2002. Conference Record of the Thirty-Sixth Asilomar Conference on
Conference_Location :
Pacific Grove, CA, USA
Print_ISBN :
0-7803-7576-9
DOI :
10.1109/ACSSC.2002.1196941
