Hpod: A High-Performance Online Deduplication Cluster

Facing the massive data rapidly growing in the data centers, deduplication storage system continuously confronts the challenges in providing the corresponding throughputs, necessary capacities to move backup data within backup and low recovery window times. One approach is to construct a well design deduplication clusters which includes many high throughputs data nodes to generate an allover throughputs up to 1.5GB/s. When we present a deduplication cluster that can deduplicate with high throughput and deduplicate rates, we focus on the routing strategy which mainly affects the performance, deduplication rates and the fault tolerance which enhance the cluster stability. We built our prototype system based on the sparing index and SRC routing strategy which is based on the chord algorithm to enhance our improvement in efficiency of deduplication, and the system introduces new techniques and structure to accelerate the IO performance.