DeDu: Building a deduplication storage system over cloud computing

Author

Sun, Zhe ; Shen, Jun ; Yong, Jianming

Author_Institution

Fac. of Inf., Univ. of Wollongong, Wollongong, NSW, Australia

fYear

2011

fDate

8-10 June 2011

Firstpage

348

Lastpage

355

Abstract

This paper presents a deduplication storage system over cloud computing. Our deduplication storage system consists of two major components, a front-end deduplication application and Hadoop Distributed File System. Hadoop Distributed File System is common back-end distribution file system, which is used with a Hadoop database. We use Hadoop Distributed File System to build up a mass storage system and use a Hadoop database to build up a fast indexing system. With the deduplication applications, a scalable and parallel deduplicated cloud storage system can be effectively built up. We further use VMware to generate a simulated cloud environment. The simulation results demonstrate that our deduplication cloud storage system is more efficient than traditional deduplication approaches.

Keywords

cloud computing; data compression; database indexing; distributed databases; storage management; DeDu; Hadoop database; Hadoop distributed file system; VMware; back-end distribution file system; cloud computing; front-end deduplication application; indexing system; mass storage system; parallel deduplicated cloud storage system; Cloud computing; Communities; Computational modeling; Fault tolerant systems; Indexes; Random access memory; Silicon compounds; Cloud storage; deduplication; efficiency; load balance;

fLanguage

English

Publisher

ieee

Conference_Titel

Computer Supported Cooperative Work in Design (CSCWD), 2011 15th International Conference on

Conference_Location

Lausanne

Print_ISBN

978-1-4577-0386-7

Type

conf

DOI

10.1109/CSCWD.2011.5960097

Filename

5960097