Optimal distribution of patterns in a heterogeneous array of transputers for backpropagation networks

Training set parallelism and network based parallelism are two popular paradigms for parallelising a feedforward (artificial) neural network. Training set parallelism is particularly suited to feedforward neural networks with backpropagation learning where the size of the training set is large in relation to the size of the network. This study analyses how we can optimally distribute the training set on a heterogeneous processor network when the number of patterns in the training set is not an integer multiple of the number of processors. It is shown that optimal allocation of patterns in such cases is a mixed integer programming problem. Using this analysis, it is found that equal distribution of training patterns among a homogeneous array of transputers is not necessarily the optimal way to allocate the patterns to processors even when the training set is an integer multiple of the number of processors