A parallel Boltzmann machine on distributed-memory multiprocessors

An efficient mapping scheme of Boltzmann machine computations onto a distributed-memory multiprocessor, which exploits the synchronous spatial parallelism, is presented. In this scheme, the neurons in a Boltzmann machine are partitioned into p disjoint sets, and each set is mapped on a processor of a p-processor system. Parallel convergence and learning algorithms of Boltzmann machines, the necessary communication pattern among the processors, and their time complexities when neurons are partitioned and mapped onto a distributed-memory multiprocessor are investigated. An expected p -processor speed-up of the parallelizing scheme over a single processor is also analyzed theoretically. This analysis can be used as a basis for determining the most cost-effective or optimal number of processors according to the given communication capabilities and interconnection topologies