WSCOM: Online Task Scheduling with Data Transfers

This paper considers the online problem of task scheduling with communication. All information on tasks and communication are not available in advance except the DAG of task topology. This situation is typically encountered when scheduling DAG of tasks corresponding to Make files executions. To tackle this problem, we introduce a new variation of the work-stealing algorithm: WSCOM. These algorithms take advantage of the knowledge of the DAG topology to cluster communicating tasks together and reduce the total number of communications. Several variants are designed to overlap communication or optimize the graph decomposition. Performance is evaluated by simulation and our algorithms are compared with off-line list-scheduling algorithms and classical work-stealing from the literature. Simulations are executed on both random graphs and a new trace archive of Make file DAG. These experiments validate the different design choices taken. In particular we show that WSCOM is able to achieve performance close to off-line algorithms in most cases and is even able to achieve better performance in the event of congestion due to less data transfer. Moreover WSCOM can achieve the same high performances as the classical work-stealing with up to ten times less bandwidth.