Load Balance Heuristics for Synchronous Iterative Applications on Heterogeneous Cluster Systems

Heterogeneous computing systems are emerging as a computing infrastructure that will enable the use of distributed heterogeneous clusters for a variety of challenging applications. The actual challenge is the load balance for tightly-coupled applications. In this paper, we focus on the important subclass of tightlycoupled applications, synchronous iterative applications and formally define their load balance problem. Two novel static meta heuristic algorithms are proposed for the load distribution: a genetic tabu hybrid search (GTHS) algorithm and a host clustering based iterative search (HCIS) algorithm, when different communication computation ratios are considered. To this end, the analysis and experiment results demonstrate the effectiveness of heuristic algorithms.