A global-local hybrid Evolutionary Strategy (ES) for Recurrent Neural Networks (RNNs) in system identification

Recurrent neural networks, through their unconstrained synaptic connectivity and resulting state-dependent nonlinear dynamics, offer a greater level of computational ability when compared with regular feedforward neural network (FFNs) architectures. A necessary consequence of this increased capability is a higher degree of complexity, which in turn leads to gradient-based learning algorithms for RNNs being more likely to be trapped in local optima, thus resulting in sub- optimal solutions. This motivates the use of evolutionary computational methods which center about the use of population- based global-search techniques as an optimization scheme. In this article, we propose the use of a hybrid evolutionary strategy (ES) approach together with an adaptive linear observer, acting as a local search operator, as a learning mechanism for general RNN applications. Illustrative examples, though largely preliminary in nature, in solving a few system identification problems, are encouraging.