Combining Fine-Grained Analysis and Scheduling to Smooth Response Time Fluctuations in Multi-tier Services

Performance, in terms of high quality of service and resource utilization for example, is one of top considerations for cloud applications. However, the response time of most multi-tier applications today frequently present large scale fluctuations (e.g., Ranging from tens of milliseconds up to tens of seconds) during periods of high resource utilization. It is important to find the reasons that cause response time fluctuations when providing good performance and high effective multi-tier systems in cloud environments. In this paper, through extensive measurements of a multi-tier application benchmark (RUBiS), we show that response time fluctuations is real and average response time is not a right measure of multi-tier services system´s performance. Through making a probing analysis of requests we find that the large scale response time fluctuations can be caused by concurrent long or mix transactions and evaluate the reason that we find is right. We also propose an effective scheduling policy called CTP(cross-tier-proportion) to smooth response time fluctuations while still achieving high resource utilization in the system.