Problems of massive parallelism in neural network simulation

Different massively parallel implementations of multilayer feedforward neural networks are presented and compared on a MasPar MP-1216, a parallel single instruction, multiple data (SIMD) computer with 16384 processors. For multilayer feedforward networks, sustained rates of up to 348 M CPS and 129 M CUPS with backpropagation are obtained, a high mark for general purpose SIMD computers. Emphasis is placed on the problems of mapping neural networks to parallel hardware, on implementation problems in obtaining high propagation rates on a SIMD machine, and on problems with the resulting learning algorithms