Deduplication with Block-Level Content-Aware Chunking for Solid State Drives (SSDs)

A critical weak point of Solid State Drives (SSDs) is the limited lifespan which stems from the characteristics of NAND flash memory. In this paper, we propose a new deduplication scheme called block-level content-aware chunking to extend the lifetime of SSDs. The proposed scheme divides the data within a fixed-size block into a set of variable-sized chunks based on its contents and avoids storing duplicate copies of the same chunk. Our evaluations on a real SSD platform show that the proposed scheme improves the average deduplication rate by 77% compared to the previous block-level fixed-size chunking scheme. Additional optimizations reduce the average memory consumption by 39% with a 1.4% gain in the average deduplication rate.