DocumentCode
2644690
Title
The effective dimension of the space of hidden units
Author
Weigend, Andreas S. ; Rumelhart, David E.
Author_Institution
Stanford Univ., CA, USA
fYear
1991
fDate
18-21 Nov 1991
Firstpage
2069
Abstract
The authors show how the effective number of parameters changes during backpropagation training by analyzing the eigenvalue spectra of the covariance matrix of hidden unit activations and of the matrix of weights between inputs and hidden units. They use the standard example of time series prediction of the sunspot series. The effective ranks of these matrices are equal to each other when a solution is reached. This effective dimension is also equal to the number of hidden units of the minimal network obtained with weight-elimination
Keywords
astronomy computing; eigenvalues and eigenfunctions; filtering and prediction theory; matrix algebra; neural nets; sunspots; time series; astronomy computing; backpropagation training; covariance matrix; effective dimension; eigenvalue spectra; hidden unit activations; hidden units; learning systems; minimal network; neural nets; sunspot series; time series prediction; weight-elimination; Covariance matrix; Eigenvalues and eigenfunctions; Parameter estimation; Psychology; Sampling methods; Statistics;
fLanguage
English
Publisher
ieee
Conference_Titel
Neural Networks, 1991. 1991 IEEE International Joint Conference on
Print_ISBN
0-7803-0227-3
Type
conf
DOI
10.1109/IJCNN.1991.170692
Filename
170692
Link To Document