Title :
The spectral radius remains a valid indicator of the Echo state property for large reservoirs
Author :
Caluwaerts, Ken ; Wyffels, Francis ; Dieleman, Sander ; Schrauwen, Benjamin
Author_Institution :
Electron. & Inf. Syst. Dept., Ghent Univ., Ghent, Belgium
Abstract :
In the field of Reservoir Computing, scaling the spectral radius of the weight matrix of a random recurrent neural network to below unity is a commonly used method to ensure the Echo State Property. Recently it has been shown that this condition is too weak. To overcome this problem, other - more involved - sufficient conditions for the Echo State Property have been proposed. In this paper we provide a large-scale experimental verification of the Echo State Property for large recurrent neural networks with zero input and zero bias. Our main conclusion is that the spectral radius method remains a valid indicator of the Echo State Property; the probability that the Echo State Property does not hold, drops for larger networks with spectral radius below unity, which are the ones of practical interest.
Keywords :
learning (artificial intelligence); recurrent neural nets; echo state property; random recurrent neural network; reservoir computing; spectral radius method; weight matrix; zero bias; zero input; Bifurcation; Mathematical model; Neurons; Recurrent neural networks; Reservoirs; Training;
Conference_Titel :
Neural Networks (IJCNN), The 2013 International Joint Conference on
Conference_Location :
Dallas, TX
Print_ISBN :
978-1-4673-6128-6
DOI :
10.1109/IJCNN.2013.6706899