Dynamical assessment of symbolic processes with backprop nets

Simple recurrent networks were trained to predict the outputs of various probabilistic symbolic process. Two of the symbolic processes made critical use of a push-down stack, and two were finite state Markov processes. The memory-intensive (stack) processes, in contrast to the Markov processes, pushed the largest Lyapunov exponents toward zero, although they never reached zero. The growth in the Lyapunov exponents was conditioned by the memory-intensiveness of thetas, not by the growth rate of the states. The results indicate a link between the traditional use of stack-memories to create complex computation and dynamical treatments of complexity based on trajectory divergence