Projection-based methods for stepsize adaptation and their application to the training of feedforward artificial neural networks

Author

Codrington, Craig W. ; Mohandes, Mohamed

Author_Institution

Dept. of Electr. Eng., Purdue Univ., West Lafayette, IN, USA

Volume

1

fYear

1994

fDate

27 Jun-2 Jul 1994

Firstpage

72

Abstract

Develops several adaptive step-size rules for gradient descent based on projecting weight and gradient vectors onto a set of unit vectors; each unit vector induces a one dimensional optimization problem which is solved by minimizing a fitted quadratic. A sum of squares criterion is then used to find the stepsize which which best fits the solution to each one dimensional optimization. The resulting stepsize rules are applied to train neural networks on parity problems of various sizes

Keywords

feedforward neural nets; learning (artificial intelligence); optimisation; vectors; feedforward artificial neural networks; gradient descent; one dimensional optimization problem; parity problems; projection-based methods; stepsize adaptation; sum of squares criterion; training; Artificial neural networks; Backpropagation algorithms; Convergence; Error correction; Feedforward neural networks; Interpolation; Neural networks; Size measurement; Stability; Time measurement;

fLanguage

English

Publisher

ieee

Conference_Titel

Neural Networks, 1994. IEEE World Congress on Computational Intelligence., 1994 IEEE International Conference on

Conference_Location

Orlando, FL

Print_ISBN

0-7803-1901-X

Type

conf

DOI

10.1109/ICNN.1994.374141

Filename

374141