A Compiler-assisted Runtime-prefetching Scheme for Heterogenous Platforms

GPGPU has been widely adopted by industry and academia. For real applications on industry, however, the data communications between CPUs and GPUs often dramatically slow down the overall performance. Another difficulty raised by GPGPU is the programming productivity. OpenMP is a high-level programming model widely accepted by industry. A software distributed shared memory system (DSM) is implemented to provide a logic shared memory space and to manage data communications between CPUs and GPUs. The DSM is block-based, and the block size is adjustable based on loop partitioning parameters. In this work, we optimize the DSM system using a compiler-assisted data-prefetching scheme. There is a prefetching thread and a prefetching worker for each sepa rated memory. The prefetching thread looks into the future, applies inter-thread use-def analysis to judge which part of the USE region has already been generated by computing threads and produces prefetching requests. The prefetching worker carries out the prefetching operations.