Effective Data Representation and Compression in Ground Data Systems

Storing vast amounts of multidimensional telemetry data presents a challenge. Telemetry data being relayed from sensors to the ground station comes in the form of text, images, audio, and various other formats. Compressing this data would optimize bandwidth usage during transmission and reduce storage resources needed at the ground level. However, the multitude of heterogeneous data types present in telemetry data and the need for data precision makes compression quite difficult. The application of a single compression technique for all data types usually yields ineffective results. We will present a telemetry data compression algorithm that utilizes Discrete Fourier Transforms (DFTs) along with different compression algorithms for different data types, including Lempel-Ziv-Welch (LZW) and Flate (which combines LZW with adaptive Huffman coding) for textual and numerical data and JPEG coding for images. Although these algorithms do not yield the greatest compression ratios, the Portable Document Format (PDF) standard supports decoding of all of them, which allows us to write our encoded data streams directly to a PDF file. This approach alleviates the need for traditional database storage systems. It also standardizes and simplifies the data retrieval, decoding, and viewing process. This work results in packets-oriented telemetry data encapsulated with multiple compression stream algorithms, which can be decoded, rendered and viewed by any standard PDF viewer. This paper presents the aforementioned algorithms and its development status as applicable proof-of-concept prototypes.