DR2: Dynamic Request Routing for Tolerating Latency Variability in Online Cloud Applications

Application latency is one significant user metric for evaluating the performance of online cloud applications. However, as applications are migrated to the cloud and deployed across a wide-area network, the application latency usually presents high variability over time. Among lots of subtleties that influence the latency, one important factor is relying on the Internet for application connectivity, which introduces a high degree of variability and uncertainty on user-perceived application latency. As a result, a key challenge faced by application designers is how to build consistently low-latency cloud applications with the large number of geo-distributed and latency-varying cloud components. In this paper, we propose a dynamic request routing framework, DR2, by taking full advantage of redundant components in the clouds to tolerate latency variability. In practice, many functionally-equivalent components have been already deployed redundantly for load balancing and fault tolerance, thus resulting in low additional overhead for DR2. To evaluate the performance of our approach, we conduct a set of experiments based on two large-scale real-world datasets and a synthetic dataset. The results show the effectiveness and efficiency of our approach.