Improving massively data parallel system performance with heterogeneity

The authors introduce a new type of combined SIMD/MIMD (single-instruction multiple-data/multiple-instruction multiple-data) architecture called a hybrid system. The hybrid system consists of two components. The first component is massively parallel and consists of a large number of slow processors that are organized in an SIMD architecture. The second component consists of only a few fast processors (possibly only one) which are organized in an MIMD architecture. The authors contend that a hybrid system provides a means to adequately adjust to the characteristics of a parallel program, i.e., changing parallelism. They describe the machine and application model, and discuss the performance impact of such a system. Viewing the CM-2 with its front-end as a special case of a hybrid system, they substantiate the arguments and report measurements for a Gaussian elimination algorithm