A general approach to gradient based learning in multirate systems and neural networks

A large class of nonlinear dynamic adaptive systems, such as dynamic recurrent neural networks, can be very effectively represented by signal-flow-graphs. Using this method, complex systems are described as a general connection of many simple components, each of them implementing a simple one-input one-output transformation, as in an electrical circuit. Following an approach originally developed by Lee (1974) for continuous-time systems based on the concept of adjoint graph, a new algorithm to estimate the derivative of the output with respect to an internal parameter have been proposed in the literature for discrete-time systems. This paper extends further this approach to multirate digital systems, which have been widely used. The new method can be employed for gradient-based learning of general multirate circuits, such as the new “multirate” neural networks