Adaptive regularization of neural networks using conjugate gradient

Author

Goutte, Cyril ; Larsen, Jan

Author_Institution

Dept. of Math. Modelling, Tech. Univ., Lyngby, Denmark

Volume

2

fYear

1998

fDate

12-15 May 1998

Firstpage

1201

Abstract

Andersen et al. (1997) and Larsen et al. (1996, 1997) suggested a regularization scheme which iteratively adapts regularization parameters by minimizing validation error using simple gradient descent. In this contribution we present an improved algorithm based on the conjugate gradient technique. Numerical experiments with feedforward neural networks successfully demonstrate improved generalization ability and lower computational cost

Keywords

adaptive systems; computational complexity; conjugate gradient methods; error analysis; feedforward neural nets; learning (artificial intelligence); adaptive regularization; computational cost; conjugate gradient; feedforward neural networks; generalization ability; gradient descent; neural networks; validation error; Computational efficiency; Convergence; Cost function; Feedforward neural networks; Feedforward systems; Iterative algorithms; Neural networks; Parameter estimation; Pattern recognition; World Wide Web;

fLanguage

English

Publisher

ieee

Conference_Titel

Acoustics, Speech and Signal Processing, 1998. Proceedings of the 1998 IEEE International Conference on

Conference_Location

Seattle, WA

ISSN

1520-6149

Print_ISBN

0-7803-4428-6

Type

conf

DOI

10.1109/ICASSP.1998.675486

Filename

675486