A fast regular expression matching engine for NIDS applying prediction scheme

Regular expression matching is considered important as it lies at the heart of many networking applications using deep packet inspection (DPI) techniques. For example, modern networking intrusion detection systems (NIDSs) typically accomplish regular expression matching using deterministic finite automata (DFA) algorithm. However, DFA suffers from the high memory consumption for the state blowup problem. Many algorithms have been proposed to compress the DFA memory storage space, meanwhile, they usually pay the price of low matching speed and high memory bandwidth. In this paper, we first propose an effective DFA compression algorithm by exploiting the similarity between DFA states. Then, we apply a next-state prediction strategy and present a fast DFA matching engine. Carefully designing the DFA matching circuit, we keep the prediction success rate by more than 99,5%, thus get a comparable matching speed with original DFA algorithm. On the side of memory consumption, experimental results show that with typical NIDS rule sets, our algorithm compressed the original DFA by more than 99%. Mapping this algorithm on Xilinx Virtex-7 FPGA chip, we get a throughput of more than 200Gbps.