Title :
A distributed backpropagation algorithm of neural networks on distributed-memory multiprocessors
Author :
Yoon, Hyunsoo ; Nang, Jong H. ; Maeng, S.R.
Author_Institution :
Dept. of Comput. Sci., Korea Adv. Inst. of Sci. & Technol., Seoul, South Korea
Abstract :
A distributed backpropagation algorithm for a fully connected multilayered neural network on a distributed-memory multiprocessor system is presented. The neurons on each layer are partitioned into p disjoint sets, and each set is mapped on a processor of a p-processor system. The algorithm, the communication pattern among the processors, and their time/space complexities are investigated, and the theoretical upper bound on speedup is obtained. The experimental speedup obtained with the algorithm on a ring of 32 transputers, which confirms the model and analysis, is reported. It is found that the choice of processor interconnection topology does not influence the speedup ratio
Keywords :
computational complexity; memory architecture; multiprocessing systems; multiprocessor interconnection networks; neural nets; backpropagation algorithm; communication pattern; distributed-memory multiprocessors; multilayered neural network; neural networks; neurons; processor interconnection topology; time/space complexities; Algorithm design and analysis; Artificial neural networks; Backpropagation algorithms; Computational modeling; Computer networks; Computer simulation; Concurrent computing; Multi-layer neural network; Neural networks; Neurons;
Conference_Titel :
Frontiers of Massively Parallel Computation, 1990. Proceedings., 3rd Symposium on the
Conference_Location :
College Park, MD
Print_ISBN :
0-8186-2053-6
DOI :
10.1109/FMPC.1990.89482
