An empirical study of L2-Boost with Echo State Networks

At the beginning of the 2000s was introduced the Echo State Network model (ESN). The model has been successfully used in temporal learning tasks. In spite of its success in practical applications, the model can present some stability problems when the parameters are not well initialized. The stability of the model is associated with the spectrum of the weight matrix. To compute the spectra is an expensive tasks when the network is large. Below, the initialization of the network parameters can depend of the benchmark problem. In this paper, we investigate the performance of the L₂-Boost procedure, one specific boosting technique, in time-series problems. We use an ensemble of ESNs which are randomly initialized without control of the spectral radius norm as weak predictors of the L₂ procedure. Therefore, the procedure consists only in a random initialization of an ensemble of ESNs following of the L₂-Boost steps.We evaluate this procedure on 5 widely used time-series benchmarks. Furthermore, we compare this procedure with a baseline approach which consists of averaging the prediction of an ensemble of ESNs with different initial network settings.