Approximation ability of a class of locally recurrent globally feed-forward neural networks

The paper investigates approximation abilities of a special class of discrete-time dynamic neural networks. These networks are called locally recurrent globally feed-forward, because they are designed with dynamic neuron models which contain inner feedbacks, but interconnections beetween neurons are strictly feed-forward ones like in the well-known multi-layer perceptron. The paper presents analytical results showing that a locally recurrent network with two hidden layers is able to approximate a state-space trajectory produced by any Lipschitz continuous function with arbitrary accuracy. Moreover, based of these results the network can be simplified and transformed to a more practical structure useful in real world applications.