Extending parallelism to memory hierarchies in massively parallel systems

The paper introduces the generalized hierarchical massively parallel system (MPS) and extends data transfer parallelism to all members of the memory hierarchy. Methods are presented for connecting multiple parallel disks and parallel memory modules to the generalized hierarchical MPS. Full utilisation of data transfer bandwidth between disks and processing elements is facilitated by structuring and corner turning the incoming data before storing it on parallel disks. One major application of MPSs is image processing. The methods presented perform well on image data. The total system is generalized by introducing architectural parameters. Varying these parameters allows the system to be represented as various forms of SIMD architecture. A new SIMD massively parallel system, BLITZEN, is used as an example to illustrate some of the concepts presented.