An Intelligent Data De-duplication Based Backup System

With the explosive growth of data amount, it has become a key issue to reduce the storage space that mass data occupies and the bandwidth consumption during network transmission. Experimental investigation shows that a large amount of redundant data exists in each part of the information processing and storage. Therefore, the issues concerning how to eliminate the redundant information during the backup process are of crucial importance for saving disk space and network bandwidth. This paper adopts the data de-duplication technology to solve the problem of redundant data in the course of backup by designing and implementing a backup system with intelligent data de-duplication named Backup Ded up which includes four de-duplication strategies, that is, SIS, FSP, CDC and SW. Backup Ded up supports the online source-side de-duplication and is capable of selecting different de-duplication algorithms according to the corresponding data types. Meanwhile, it offers the data reliability and security in the backup process. The experimental test results show that Backup Ded up employs multi de-duplication strategies simultaneously to substantially eliminate redundant data in the backup process so as to reach the goal of effectively saving storage space and network bandwidth.