Effects of weight discretization on the back propagation learning method: algorithm design and hardware realization

An architectural configuration for the back-propagation (BP) algorithm is illustrated. The circuit solution for the basic blocks is presented, and the effect of weight discretization on the BP algorithm is analyzed. It is demonstrated, through simulations, how the BP algorithm can be operated successfully with discretized weights. In particular, better performances can be achieved with an exponential discretization, i.e. the strength of weights varies exponentially with the controlling variable (voltage). The discretized voltage values differ by a quantity high enough that the neural network can be backed up with a refresh technique in combination with a multilevel dynamic memory that entails a particularly low wiring cost. A quasi-analog adaptive architecture is devised, properly matching the BP algorithm, and its CMOS circuit implementation is detailed. The mechanism controlling weight changes is simple enough to be reproduced locally at each synapsis site, thus meeting one of the requirements for an efficient storage technology for analog VLSI