An active scheduler: autonomous concurrency control of parallel programs in a distributed environment

We propose a new scheduling method that can simultaneously achieve two main goals of task scheduling in distributed parallel systems; i.e., to minimize the execution time of a parallel job without disturbing the execution of other jobs. We challenge to achieve those goals by introducing a new scheduler, called active scheduler, that controls the priority of parallel programs dynamically and balances the workload of computers, depending on the current status of the runtime environment. The priority of parallel programs is controlled by controlling the concurrency of the programs. We implemented a prototype system to evaluate the effectiveness of active scheduler. The results of experiments imply that the overhead of introducing the active scheduler is bounded by 15% of the original execution time, and it is in fact effective to adjust the execution of parallel programs to an actual distributed parallel processing environment in which many users execute their jobs at the same time