Adaptive scheduling of workflows on multicluster platforms

Recent advances in parallel and distributed computing have made it very challenging for programmers to reach the performance potential of current systems. In addition, recent advances in numerical algorithms and software optimizations have tremendously increased the number of alternatives for solving a problem, which further complicates the software tuning process. Indeed, no single algorithm can represent the universal best choice for efficient solution of a given problem on all compute substrates. In this paper, we address the problem of scheduling of scientific workflows on multicluster platforms composed of clusters of clusters. More specifically, given multiple choices for solving a particular problem, we develop a poly-algorithm which determines the choice expected to perform the best at the specific setting depending on problem and platform characteristics. Simulation results showed that the poly-algorithm provide interesting performance for different execution scenarios.