A systolic algorithm for back-propagation: mapping onto a transputer network

The paper is devoted to the implementation of Back-Propagation (BP) on local memory multiprocessor systems (LMM). First, a systolic algorithm (SA) is described, where dependencies are considered at the data item level. Next, the systolic array is partitioned and mapped onto a multiprocessor system. At this stage, the level of granularity is increased, in order to reduce communication cost. Finally, each stage is implemented on a transputer based multiprocessor, and their performance is compared with a simple sequential version of the learning rule. A parallelization rate of about 0.9 is obtained. Back-Propagation is a supervised, gradient descent learning rule for multilayered, feed-forward connectionist networks