A Scalable Linkage Method for Large Scale Event-Data Processing

In this paper, we propose a new linkage method for the great amount of event data in distributed computing environments. This method is used for event-data processing such as System Behavior Visualization, which can diagnose and provide information about complex distributed IT system behavior. The event data using System Behavior Visualization are network messages between IT system servers. In this method, we use two techniques: probe and data distribution by "source". The probe is a useful post-processing tool when event data have come fast and finding the linkable event data has been difficult due to limited time. For the probe method, the counts of probe communication are very important for scalability. In our method, event data are allocated based on their "source" information because this reduces the counts of probe communication. We evaluate our method from the view-points of the throughput related with the load and the number of event processing servers. By using this method, the event-data processing can perform highly in the distributed computing environment. Experimental data demonstrated the viability of our propose method.