A user-centric dynamic cluster partitioning approach for HPC service optimization

In this paper, we study how resources within a large high performance computing (HPC) cluster can be dynamically partitioned to optimize client utility for multiple service classes. We model service effectiveness using both perceived service quality and resources required. Using empirical data obtained from A*STAR Computational Resource Center (A*CRC), we analyze how quality metrics and statistical characteristics of HPC jobs affect user satisfaction. We derive the optimal number of processors required to achieve the maximal overall client utility in M/G/1 based clusters. Based on measured job characteristics, we propose a statistics-based client utility optimization (SCUO) algorithm, which dynamically partitions the cluster into resource groups serving different service classes. Simulations show that our proposed algorithm is able to achieve better performance with both higher client utility and higher job admission rates.