Fast Linear Algebra on GPU

GPUs have been successfully used for acceleration of many mathematical functions and libraries. A common limitation of those libraries is a minimal size of primitives being handled in order to achieve significant speedups compared to their CPU versions. The minimal size requirement can prove prohibitive for many applications. It can be loosened by batching operations to have sufficient amount of data to perform calculations maximally efficiently on the GPU. A fast OpenCL implementation of two basic vector functions-vector reduction and vector scaling-is described in this paper. Its performance is analyzed by running benchmarks on two of the most common GPUs in use-Tesla and Fermi NVIDIA GPUs. Reported experimental results show that our implementation significantly outperforms the current state-of-the-art GPUbased basic linear algebra library CUBLAS.