Issues in scalable library design for massively parallel computers

The authors examine some critical issues raised in the design of libraries for MPCs, such as scalability, portability, recompilation, and flexibility. They advocate a layered structure of library design, comprising a high-level language layer, a machine-independent node layer, a machine-dependent node layer, and an object code layer for different demands and requirements. They discuss the impact of various data decomposition strategies on program performance and the computation and communication analysis techniques used at different layers. They also propose the concept of the range of scalability as a metric for selecting the most appropriate implementation. A linear system solver based on the Gaussian elimination method is used as an example to illustrate various design alternatives.