A New Data Streaming Method for Locating Hosts with Large Connection Degree

Locating hosts with large connection degree is very important for monitoring anomalous network traffics. The in-degree (out-degree), defined as the number of distinct sources (destinations) that a network host is connected with (connects) during a given time interval. Due to massive amount of data in high speed network traffics and limit on processing capability, it is difficult to accurately locate hosts with large connection degree over high speed links on line. In this paper we present a new data streaming method for locating hosts with large connection degree based on the reversible connection degree sketch to monitor anomalous network traffics. The required memory space is small and constant, and more importantly the update/query complexity would not depend on the amount of data. The hash functions for data sketch are designed based on the remainder characteristics of the number theory so that in-degree/out-degree associated with a given host can be accurately estimated. Although the connection degree sketch does not preserve any host address information, we can analytically reconstruct the host addresses associated with large in-degree/out-degree by a simply equation purely based on the characteristics of the hash functions without using any host address information. This procedure is highly efficient since the computational time is constant and ignorable. Furthermore, this reversible connection degree sketch based method can be easily implemented in distributed systems. The experimental and testing results based on the actual network traffics show that the new method is truly accurate and efficient.