A digital neural network VLSI with on-chip learning using stochastic pulse encoding

A digital neural network VLSI chip, RN200 has been developed and fabricated. Sixteen neurons and totally 256 synapses are integrated in a 13.73×13.73 mm² VLSI chip, fabricated by RICOH 0.8 μm CMOS technology. Multiple-layer neural network can be made by combining two or more-chips. Signals within the network (e.g., activations, error signals, connection weights) are represented by stochastic digital pulse trains. Both feed forward and learning processes are efficiently implemented with simple logical gates. Our novel approach for approximating the derivative of activation function is described. The approximation circuit requires only a few gates. Multiple-RNG architecture is adopted to ensure the random distribution of pulses. Both seeds and configurations of the random number generators on the chip can be updated dynamically and randomly by this mechanism. The effectiveness of the derivative and the Multiple-RNG architecture are simulated and verified with the learning performance in a hand-written character recognition problem. The chip can perform 5.12 gigapulse operations per second. It corresponds to effective neural computing rate of 40M CPS or 40M CUPS.