Andes: A Highly Scalable Persistent Messaging System

A combination of factors: expanding user bases, the ubiquity of mobile communications, and newer technologies such as cloud computing and multi-core computing, are pushing todays systems to grow larger and larger. With their loosely coupled nature, distributed messaging systems often play a key role in such architectures. However, just like other parts of the architecture, those messaging systems also need to scale up, and they need to do so in three dimensions: quantity of messages, number of users, and size of messages. Although most current systems handle the first two dimensions, few of them efficiently support the third dimension. This paper proposes a novel method to implement a scalable and persistent broker that supports a publish/subscribe model and distributed queues using a NoSQL database and a coordination framework. We will discuss the design that uses recent advances in scalable database management and distributed coordination middleware, and we will compare the proposed models with other distributed message brokers.