Trajectory engine: a backend for trajectory sampling

The management of communication networks increasingly requires detailed knowledge of network usage, acquired by direct measurement. We report on the design and implementation of a backend system for trajectory sampling, a method for consistent sampling of packets across a network domain. This trajectory engine collects trajectory samples and stores them after appropriate preprocessing. It provides a querying and visualization tool to aid in traffic engineering and troubleshooting. We describe the entire system, and in particular the design choices that we took in order to balance the scale of the system (due to large volumes of measured data) with resource usage while providing useful functionality for users. In the preprocessing stage, we focus on reassembly of trajectories from individual samples. We describe the design of the database for efficient storage of trajectories, and its relationship with the query and visualization interface. We test the system using a synthetic stream of trajectory samples derived from configuration and usage data from the network of a major service provider. We walk through several examples that illustrate how a network operator might take advantage of trajectory sampling through such a tool.