Title :
The effective dimension of the space of hidden units
Author :
Weigend, Andreas S. ; Rumelhart, David E.
Author_Institution :
Stanford Univ., CA, USA
Abstract :
The authors show how the effective number of parameters changes during backpropagation training by analyzing the eigenvalue spectra of the covariance matrix of hidden unit activations and of the matrix of weights between inputs and hidden units. They use the standard example of time series prediction of the sunspot series. The effective ranks of these matrices are equal to each other when a solution is reached. This effective dimension is also equal to the number of hidden units of the minimal network obtained with weight-elimination
Keywords :
astronomy computing; eigenvalues and eigenfunctions; filtering and prediction theory; matrix algebra; neural nets; sunspots; time series; astronomy computing; backpropagation training; covariance matrix; effective dimension; eigenvalue spectra; hidden unit activations; hidden units; learning systems; minimal network; neural nets; sunspot series; time series prediction; weight-elimination; Covariance matrix; Eigenvalues and eigenfunctions; Parameter estimation; Psychology; Sampling methods; Statistics;
Conference_Titel :
Neural Networks, 1991. 1991 IEEE International Joint Conference on
Print_ISBN :
0-7803-0227-3
DOI :
10.1109/IJCNN.1991.170692
