Optimization of parallel BP implementation: training speed of 1056 MCUPS on the massive

Author

Yasunaga, Moritoshi ; Yoshida, Eiji

Author_Institution

Inst. of Inf. Sci. & Electron., Tsukuba Univ., Ibaraki, Japan

Volume

1

fYear

1998

fDate

4-8 May 1998

Firstpage

563

Abstract

For the backpropagation (BP) implementation on parallel computers, the hybrid approach of the data and the node parallelization techniques has been widely used to utilize the target computers efficiently. However, nothing related to the optimization technique for that hybrid parallelization has been explored yet. In this paper, we discuss the approach theoretically and propose an optimization technique. Experiments were carried out on the recently developed parallel computer CP-PACS. We show that the experimental results agree well with the theoretical predictions. By using the optimization technique, the maximum training speed of 1056 MCUPS (million connections updated per second) has been achieved

Keywords

backpropagation; neural nets; optimisation; parallel architectures; parallel machines; CP PACS parallel computers; backpropagation; learning; neural nets; node parallelization; optimization; Acceleration; Computer architecture; Computer networks; Concurrent computing; Impedance; Network topology; Neural networks; Neurons; Parallel processing; Switches;

fLanguage

English

Publisher

ieee

Conference_Titel

Neural Networks Proceedings, 1998. IEEE World Congress on Computational Intelligence. The 1998 IEEE International Joint Conference on

Conference_Location

Anchorage, AK

ISSN

1098-7576

Print_ISBN

0-7803-4859-1

Type

conf

DOI

10.1109/IJCNN.1998.682329

Filename

682329