Self-Adaptive Scheduler Parameterization via Online Simulation

Although thoroughly investigated, job scheduling for high-end parallel systems remains an inexact science, requiring significant experience and intuition from system administrators to properly configure batch schedulers. Production schedulers provide many parameters for their configuration, but tuning these parameters appropriately can be very difficult - their effects and interactions are often nonintuitive. In this paper, we introduce a methodology for automating the difficult process of job scheduler parameterization. Our proposed methodology is based on using past workload behavior to predict future workload, and on online simulations of a model of the actual system to provide on-the-fly suggestions to the scheduler for automated parameter adjustment. Detailed performance comparisons via simulation using actual supercomputing traces indicate that out methodology consistently outperforms other workload-aware methods for scheduler parameterization.