Developing software for parallel computing systems Original Research Article

In recent years parallel computing systems based on a collection of a modest number (generally in the range of 16 to 1024) relatively high powered processors (such as those used in scientific workstations) have become commercially available. The current systems software for these computing systems requires that programmers use special techniques to benefit from the improved computing speed of the systems. When converting programs that have been written for conventional sequential or vector systems to one of the new parallel systems, programmers must be aware of the paradigms of parallel computing and the primitive operations used by the new system. This paper lays the foundation for converting old code to parallel code by describing two paradigms of parallel computing: data parallel and message passing. Some primitive operations of these two paradigms are described and illustrated by examples from existing languages, and a few examples of data parallel code are given.