Impact of Job Mix on Optimizations for Space Sharing Schedulers

Abstract Modern parallel systems with N nodes can concurrently service multiple jobs requesting a total of up to to N nodes. One of the challenges for the operating system is to give reasonable service to a diverse group of jobs. Asequence of large jobs, each requiring over half of the available nodes, can reduce the machine utilization by up to 50%, but scheduling a long running job on the idle nodes may block the stream of large jobs. Various policies have been proposed for scheduling parallel computers, but as the users of current supercomputers know, these policies are far from perfect. This paper reports on the measurement of the usage of a 512-node IBM SP2 at Cornell Theory Center, a 96-node Intel Paragon at ETH Zurich, and a 512-node Cray T3D at Pittsburgh Supercomputing Center. We discuss the characteristics of the different workloads and examine their impact on job scheduling. We specifically show how two simple scheduling optimizations based on reordering the waiting queue can be used effectively to improve scheduling performance on real workloads. Supercomputer workloads from different installations exhibit some common characteristics, but they also differ in important ways We demonstrate how this knowledge can be exploited in the design and tuning of schedulers.