Poster: The Magic Determination of the Magic Constants by ttgLib Autotuner

When the program is being optimized for execution on GPU, one has to introduce a lot of performance affected constants that define blocks parameters, data chunks size, parallelism granularity, etc. And the more software is optimized, the more magic constants it introduces. Furthermore, adding multi-GPU system support often requires usage of smart load balancing strategies that considers GPU-specific effects such as potential speed-up from ignoring some accelerators, time volatility of GPU-performance and others. As a result, performance of the target software can be significantly increased just by tuning to the hardware and processing data being used. The authors developed a means of determining the optimal values of these constants called ttgLib autotuner which is capable of monitoring the software at runtime and automatically tuning magic constants as well as performing dynamic load balancing between CPU and multiple GPUs. The performed tests showed the additional speedup upto 50-80% by tuning alone.