Adaptive in-network query deployment for shared stream processing environments

In-network processing has been an important research problem for distributed stream processing systems. In this paper, we study the problem in the context of shared processing environments, where query operators could be used by multiple applications. In these environments, run-time modifications on the query deployment add new challenges. Applications with strict and potentially conflicting QoS requirements may share operators. Hence, operator placement decisions must be fast, adaptive to network conditions, and well-coordinated in order to guarantee the QoS expectations. We propose a novel sharing-aware middleware for in- network processing that achieves fast adaptivity to dynamic changes. We follow a proactive approach where nodes propagate metadata regarding alternative operator placement configurations. Whenever QoS violations occur, the metadata enables nodes to make fast, localized, operator migration decisions that can adapt to dynamic network conditions and resolve the existing violations.