Load distribution for multi-way streams joins using cluster

As the development of sensor, network, etc., more and more data are supplied in the form of streams. In most cases, it is essential to join multi-way streams in order to get potential information in real time. Unfortunately, as the volume of data increasing, the capability of single node model could not catch up with the application requirements. Executing the join process in a shared-nothing cluster is one of practical solutions. How to routing the streams to improve the throughput and guarantee the load balance in the cluster is an important problem. In this paper, we present a method named Time Segment policy to routing data streams in cluster. Our method can guarantee (1) the related tuples are routed to the same nodes; (2) load balance among all computing nodes under the promise of (1). The experiments show that our method can distribute streams effectively in the “master-worker architecture” cluster model.