Scalable and Efficient End-to-End Network Topology Inference

To construct an efficient overlay network, the information of underlay is important. We consider using end-to-end measurement tools such as traceroute to infer the underlay topology among a group of hosts. Previously, Max-Delta has been proposed to infer a highly accurate topology with a low number of traceroutes. However, Max-Delta relies on a central server to collect traceroute results and to select paths for hosts to traceroute. It is not scalable to large groups. In this paper, we investigate a distributed inference scheme to support scalable inference. In our scheme, each host joins an overlay tree before conducting traceroute. A host then independently selects paths for tracerouting and exchanges traceroute results with others through the overlay tree. As a result, each host can maintain a partially discovered topology. We have studied the key issue in the scheme, that is, how a low-diameter overlay tree can be constructed. Furthermore, we propose several techniques to reduce the measurement cost for topology inference. They include 1) integrating the Doubletree algorithm into our scheme to reduce measurement redundancy, 2) setting up a lookup table for routers to reduce traceroute size, and 3) conducting topology abstraction and reducing the computational frequency to reduce the computational overhead. As compared to the naive Max-Delta, our scheme is fully distributed and scalable. The computational loads for target selection are distributed to all the hosts instead of a single server. In addition, each host only communicates with a few other hosts. The consumption of edge bandwidth at a host is hence limited. We have done simulations on Internet-like topologies and conducted measurements on PlanetLab. The results show that the constructed tree has a low diameter and can support quick data exchange between hosts. Furthermore, the proposed improvements can efficiently reduce measurement redundancy, bandwidth consumption, and computational overhead.