Supporting Distributed Application Workflows in Heterogeneous Computing Environments

Next-generation computation-intensive applications in various fields of science and engineering feature large-scale computing workflows with complex structures that are often modeled as directed acyclic graphs. Supporting such task graphs and optimizing their end-to-end network performances in heterogeneous computing environments are critical to the success of these distributed applications that require fast response. We construct analytical models for computing modules, network nodes, and communication links to estimate data processing and transport overhead, and formulate the task graph mapping with node reuse and resource sharing for minimum end-to-end delay as an NP-complete optimization problem. We propose a heuristic approach to this problem that recursively computes and maps the critical path to the network using a dynamic programming-based procedure. The performance superiority of the proposed approach is justified by an extensive set of experiments on simulated data sets in comparison with existing methods.