Algorithm for redundancy elimination in network traffic

The network redundant traffic, not only consuming network bandwidth, but also reducing the efficiency of the Internet, exists abundantly in the course of exchanging Internet data. In the context, protocol-independent redundancy elimination, which can remove duplicate strings from within arbitrary network flows, has emerged as a powerful technique to improve the efficiency of network links in the face of repeated data and has been widely used. Redundancy elimination middle-boxes are being widely deployed to improve the effective bandwidth of network access links of enterprises and data centers alike, and for improving link loads in small ISP networks.