Communication issues in parallel Conjugate Gradient method using a star-based network

Conjugate Gradient (CG) method is an iterative linear solver which is used by many scientific and engineering applications to solve a linear system of algebraic equations. CG generates a heavy load of computation and therefore it slows the performance of the applications using it. Parallelizing CG is considered as a way to increase its performance. However, CG suffers from communication dependencies among its divisible loads. Most of the studies to parallelize CG concentrate on parallelizing its matrix-vector multiplication. In this paper, we answer the following questions: 1) what are the divisible loads in the CG, and 2) where is communication involved in the parallel CG. To answer 1), we highlight the different divisible data blocks in CG. To answer 2), we introduce a dependency graph among the different data blocks. We conduct experiments on a parallel CG implementation and evaluate communication cost.