Learning Cyclic Oscillation By Digital Type Recurrent Neural Network

The error back propagation through time (called BPTT) is a learning method of the recurrent neural network. The network learned by BPTT can solve the dynamical problem with the time series data. However, it is not able to directly solve a digital type problem where the middle layer output is intrinsically binary such as the internal state inference of the automaton. Oohori et al. proposed a digital version of the back propagation (called DBP) for a hierarchical digital neural network consisting of the binary units. The DBP can solve a linearly nonseparable problem, and learning performance is comparable to the conventional BP. We propose a digital back propagation through time (called DBPTT) based on the DBP where the BPTT is applied to the digital type recurrent neural network. The learning of the DBPTT is fast and easy for hardware implementations. Simulation results for digital cyclic oscillation show that the performance of the DBPTT is comparable in learning and superior in generalization to the conventional BPTT.