Correlations in end-to-end network metrics: Impact on large scale network monitoring

With the ever growing size of the Internet and increasing popularity of the overlay and peer-to-peer networks, scalable end-to-end (e2e) network monitoring is essential for better network management and application performance. For large scale networks, an e2e monitoring infrastructure should minimize the measurement cost while ensuring that the network is still monitored at fine enough time-scales required for each application flow. We explore the relationships between different e2e network metrics with the aim of leveraging such relationships for reducing monitoring costs while maintaining measurement accuracy. We analyze long range network measurements from PlanetLab, where we collected e2e network data (route, number of hops, capacity bandwidth and available bandwidth) for about two years on several thousand paths. We also present a few schemes to leverage the metric correlations and reduce the monitoring cost. Our preliminary results indicate that in some cases, we can reduce the monitoring costs by 75% while maintaining the accuracy at about 88%.