A robust application-level approach for responsiveness differentiation

There is a growing demand for provisioning of proportional responsiveness differentiation to various clients on scalable Web servers to meet changing resource availability, and to satisfy different client requirements. Theoretically, a queueing-based processing rate allocation scheme is able to achieve the objective by providing different processing rates to requests of different client classes. However, we find that an implementation of the queueing-theoretical scheme shows weak proportionality with large variance because it does not have fine-grained control over the resources that the kernel consumes and hence the processing rate is not strictly proportional to the number of processes allocated. We design a feedback controller and integrate it with the queueing-theoretical scheme. The integrated application-level approach allocates a certain number of processes to handle requests of different client classes according to the queueing-theoretical scheme. The process allocations are then adjusted according to the difference between target response time and the achieved response time by using proportional integral derivative control. Results demonstrate that this integrated approach can enable Web servers to provide fine-grained response time differentiation. The approach is robust and can be practically deployed on Apache Web servers.