An Adaptive Scheduling Policy for Staged Applications

The performance of Web servers and application servers is a crucial factor for the success of the underlying business activity. Current commercial servers (such as Apache and Microsoftpsilas IIS) usually employ a thread-based concurrency policy for executing request simultaneously. However, with this policy the throughput tends to collapse when massive overload occurs. Event-based concurrency has been proposed as an alternative, but - similar to the thread-based approach - it suffers in its pure form from a potential memory access bottleneck. This is avoided by a server based on a staged event-driven architecture (SEDA) which combines the two approaches to take advantage of locality in data and code within each state. In this paper, we present a three layer observer/controller architecture for dynamic performance control of SEDA-based application servers. Control is achieved by a strategy which adapts the resources assigned to each stage based on the observed throughput and queue length of each stage using a feedback loop. To validate our strategy and to compare with other strategies, we implemented a simulation environment. The results gained demonstrate that our approach successfully self-optimizes the CPU time allocation and achieves a competitive system throughput while avoiding performance degradation under overload and dynamic changes in the system.