Data Compression Cost Optimization

This paper proposes a general optimization framework to allocate computing resources to the compression of massive and heterogeneous data sets incident upon a communication or storage system. The framework is formulated using abstract parameters, and builds on rigorous tools from optimization theory. The outcome is a set of algorithms that together can reach optimal compression allocation in a realistic scenario involving a multitude of content types and compression tools. This claim is demonstrated by running the optimization algorithms on publicly available data sets, and showing up to 25% size reduction, with equal compute-time budget using standard compression tools.