Feed-forward echo state networks

New method for modeling nonlinear systems called the echo state networks (ESNs) has been proposed recently by H. Jaeger and H. Haas (2004). ESNs make use of the dynamics created by huge randomly created layer of recurrent units. Dynamical behavior of untrained recurrent networks was already explained in the literature and models using this behavior were studied by J.F. Kolen (1994) and by P. Tino et al. (1998). They are based on the fact that the activities of the recurrent layer of the recurrent network randomly initialized with small weights reflect history of the inputs presented to the network. Knowing how the recurrent layer stores the information and understanding the state dynamics of recurrent neural networks we propose modified ESN architecture. The only "true" recurrent connections are backward connection from output to recurrent units and the reservoir is built only by "forwardly" connected recurrent units. We show that this simplified version of the ESNs can also be successful in modeling nonlinear systems.