Weight groupings in the training of recurrent networks

Author

Chan, Lai-Wan ; Szeto, Chi-Cheong

Author_Institution

Comput. Sci. & Eng. Dept., Chinese Univ. of Hong Kong, Shatin, Hong Kong

Volume

3

fYear

2000

fDate

2000

Firstpage

21

Abstract

We use the block-diagonal matrix to approximate the Hessian matrix in the Levenberg Marquardt method for the training of recurrent neural networks. Substantial improvement of the training time over the original Levenberg Marquardt method is observed without degrading the generalization ability. Three weight grouping methods, correlation blocks, k-unit blocks and layer blocks were investigated and compared. Their computational complexity, approximation ability, and training time are analyzed

Keywords

Hessian matrices; computational complexity; generalisation (artificial intelligence); learning (artificial intelligence); recurrent neural nets; Hessian matrix; Levenberg Marquardt method; approximation; block-diagonal matrix; computational complexity; correlation blocks; generalization; learning time; recurrent neural networks; weight grouping; Approximation methods; Computer science; Degradation; Equations; Intelligent networks; Jacobian matrices; Learning systems; Matrix decomposition; Neurons; Recurrent neural networks;

fLanguage

English

Publisher

ieee

Conference_Titel

Neural Networks, 2000. IJCNN 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on

Conference_Location

Como

ISSN

1098-7576

Print_ISBN

0-7695-0619-4

Type

conf

DOI

10.1109/IJCNN.2000.861275

Filename

861275