A simplex optimization approach for recurrent neural network training and for learning time-dependent trajectory patterns

A major problem in a successful deployment of recurrent neural networks in practice is the complexity of training due to the presence of recurrent and feedback connections. The problem is further exacerbated if gradient descent learning algorithms that require computation of error gradients for the necessary updating are used, often forcing one to resort to approximations that may in turn lead to reduced training efficiency. We describe a learning procedure that does not require gradient evaluations and hence offers significant implementation advantages. This procedure exploits the inherent properties of nonlinear simplex optimization in realizing these advantages