Look before You Leap: Using the Right Hardware Resources to Accelerate Applications

GPUs are widely used to accelerate data-parallel applications. However, while the GPU processing capability is enhanced in each generation, the CPU computing power is also increased by adding more cores and widening vector units. Compared to the rapid development of GPUs and CPUs, the bandwidth of the data transfer between GPUs and the host CPU grows much slower, resulting in a data-transfer wall for using GPUs. In this situation, choosing the right mix of hardware resources - i.e., The right hardware configuration - is critically important for improving application performance, and the right choice is a function of the available hardware resources as well as the application and the dataset to be used. In this paper, we present a systematic approach to determine the hardware configuration that leads to the best performance for a given workload. Our approach captures the variation of hardware capabilities and data-transfer overhead for different applications and datasets, and uses modeling and prediction techniques to determine the best-performing hardware configuration. We have tested our approach on 7 applications with 6 datasets per application. The results show that our approach takes the correct decision in 38 out of 42 test cases, achieving up to 12.6×/6.6× performance improvement compared to an uninformed Only-CPU/Only-GPU decision.