Load Balancing for Performance Differentiation in Dual-Priority Clustered Servers

Size-based policies have been known to successfully balance load and improve performance in homogeneous cluster environments where a dispatcher assigns a job to a server strictly based on the job size. We first examine how size-based policies can provide service differentiation and complement admission control and/or priority scheduling policies. We find that under auto correlated arrivals the effectiveness of size-based policies quickly deteriorates. We propose a two-step resource allocation policy that makes resource assignment decisions based on the following principles. First, instead of equally dispatching the work among all servers in the cluster, the new policy biases load balancing by an effort to reduce performance loss due to autocorrelation in the streams of jobs that are directed to each server. As a second step, an additional, per-class bias guides resource allocation according to different class priorities. As a result, not all servers are equally utilized (i.e., the load in the system becomes unbalanced) but performance benefits are significant and service differentiation is achieved as shown by detailed trace-driven simulations