Inherent Replica Inconsistency in Cassandra

Inherent replica inconsistency refers to the difference among the replicas of the same logical data item in the write propagation process of a normally running distributed storage system. In this paper, we formalize the write propagation process model of Cassandra, a widely used NoSQL storage system. In the write propagation process we explore two queueing systems, sending task queues and mutation queues, which locate at each replica node and are determinants of the replica inconsistency. The departure time difference from the mutation queue is used as the measure of inconsistency between two replicas. Furthermore, Request Per Second (RPS) and Mutation Threads Number (MTN), which affect the inherent inconsistency, are discussed and the MTN adaptation algorithm is proposed. Finally, A Cassandra inconsistency measurement framework is implemented using the source instrumentation approach. The empirical results conform well with our proposed inconsistency measurement model.