Leveraging Correlations between Capacity and Available Bandwidth to Scale Network Monitoring

Recently, there has been a tremendous growth in the number of installed distributed computing platforms such as those for content distribution networks, cloud computing infrastructures, and distributed data centers. Such distributed platforms need a scalable end-to-end (e2e) network monitoring component to provide Quality of Service (QoS) guarantees to the services and improve the overall performance. An important challenge for a network monitoring infrastructure is the periodicity of the measurements as this aspect trades off the monitoring overheads with staleness of the results. In the Network Genome project, we explore the relationships between different e2e network metrics with the aim of leveraging such relationships for reducing monitoring costs while maintaining measurement accuracy. We perform our analysis using long range network measurements from PlanetLab, where we have been collecting e2e network data (route, number of hops, capacity bandwidth and available bandwidth) as part of the S3 system since January 2006. In this paper, we focus on the correlation between the Capacity and Available Bandwidth metrics between host pairs in the PlanetLab testbed. Our analysis shows that the ranking of hosts with respect to their Capacity to/from a set of nodes is a good indicator of the ranking of hosts with respect to their Available Bandwidth to/from the same set of nodes.