A self-recovery technique for highly-available stream processing over local area networks

We present a replication-based and self-recovery-based approach, replica backup, that realizes both continuous and highly-available data stream processing over local area networks. In our approach, we use process-pairs mechanism in which peer operators run in parallel and independently so that each downstream operator can use whichever data arrives first. To further realize continuously stable communication among operators and improve the robustness of system, we devise automatical recovery mechanism that overcomes the limitation of one-off recovery mechanism. In this paper, we first outline the basic design and framework associating with our self-recovery technique. Next, we develop central leader election algorithm (CLEA) that can choose a new operator according with the placement of candidates. This operator placement algorithm that directly measures the latency among operators aims to balance the cost of data stream processing and latency guarantee. Finally, we compare our replica backup method with previous high-available technique through experiments on network simulator ns-3 to demonstrate the utility of our work.