An efficient storage technique for network monitoring data

Monitoring modern networks involves storing and transferring huge amounts of data. For this reason, compression techniques are typically used in order to reduce the space and time needed for these operations. The main drawback of this approach is that, when data has to be processed, a preliminary decompression is necessary, which increases the time and computational power needed. To cope with this problem, in this paper we propose a technique that allows to transform the measurement data in a representation format meeting two main objectives at the same time. Firstly, it allows to perform a number of operations directly on the transformed data with a controlled loss of accuracy, thanks to the mathematical framework it is based on. Secondly, the new representation has a small memory footprint, allowing to reduce the space needed for data storage and the time needed for data transfer. To validate our technique, we perform an analysis of its performance in terms of accuracy and memory footprint. The results show that the transformed data closely approximates the original data (within 5% relative error) while achieving the compression ratio of 20%; storage footprint can be gradually made close to that of the state-of-the-art compression tools, such as bzip2, if higher approximation is allowed.