Toward data distribution independent parallel matrix multiplication

To eliminate or reduce initial data redistribution overheads for distributed memory parallel computers, this paper considers the problem of writing data distribution independent (DDI) programs whose functionality and execution time are independent of initial data distributions. Relations between time-space mappings and input data distributions are established. These relations are the basis of a systematic approach to the derivation of DDI programs which is illustrated for matrix-matrix multiplication. Conditions on data distributions that correspond to an optimal modular mapping are provided. It is shown that only twelve programs suffice to accomplish redistribution-free execution for the many input data distributions that satisfy the above conditions. When DDI matrix multiplication programs are used in an algorithm with multiple matrix products, half of data redistributions otherwise required can be eliminated