Building a Robust and Efficient Middleware for Geo-replicated Storage Systems

In order to meet the needs of increasing users and improve user-perceived latency, online services distribute and replicate data across geographically diverse data centers and direct user requests to the closest or least loaded server. Distributed Hash Table (DHT) is a structured overlay network that is widely utilized in geo-replicated storage systems, e.g., Dynamo. Some geo-replicated storage systems may need to locate an item with only keywords. In this paper, we present Jupiter, a DHT-based middleware system for building geo-replicated storage systems. Jupiter provides robust and efficient routing mechanisms under geo-replicated environments. The key innovation in Jupiter is the integration of two concepts: robustness and efficiency. We have prototyped Jupiter, deployed it on a network of Linux machines, and used it to develop several distributed applications. We confirm the practicality, effectiveness and efficiency of Jupiter by conducting an extensive performance benchmark measured by efficiency, robustness and consistency.