The Challenges of Intrusion Detection Compression Technology

Database management system (DBMS) controls and manages data to eliminate data redundancy and to ensure integrity, consistency and availability of the data, among other features. Even though DBMS vendors continue to offer greater automation and simplicity in managing databases, the need for specialized intrusion detection database compression technology has not yet been addressed. Our research focuses on developing such technology. The focus is not only on compression but also on database management through planning and best practice adoption to improve operational efficiency, and provide lower costs, privacy and security. The focus in this summary is on the compression part of the DMBS system for intrusion detection. We present a methodology employing grammar-based and large alphabet compression techniques which involves the generation of multiple dictionaries for compressing clustered subfiles of a very large data file. One of the dictionaries is a common dictionary which models features common to the subfiles. In addition, non-common features of each subfile are modeled via an auxiliary dictionary. Each clustered subfile is compressed using the augmented dictionary consisting of the common dictionary together with the auxiliary dictionary for that subfile.