Title :
Software-based erasure codes for scalable distributed storage
Author :
Cooley, Joseph A. ; Mineweaser, Jeremy L. ; Servi, Leslie D. ; Tsung, Eushiuan T.
Author_Institution :
Lincoln Lab., MIT, Lexington, MA, USA
Abstract :
This paper presents a new class of erasure codes, Lincoln Erasure codes (LEC), applicable to large-scale distributed storage that includes thousands of disks attached to multiple networks. A high-performance software implementation that demonstrates the capability to meet these anticipated requirements is described. A framework for evaluation of candidate codes was developed to support in-depth analysis. When compared with erasure codes based on the work of Reed-Solomon and Luby (2000), tests indicate LEC has a higher throughput for encoding and decoding and lower probability of failure across a range of test conditions. Strategies are described for integration with storage-related hardware and software.
Keywords :
computer network reliability; fault tolerant computing; file servers; hard discs; parity check codes; redundancy; system recovery; Lincoln Erasure codes; decoding; encoding; failure probability; high-performance software implementation; large-scale distributed storage; multiple networks; scalable distributed storage; software-based erasure codes; storage-related hardware; storage-related software; Computer architecture; Knowledge management; Laboratories; Moore´s Law; Optical fiber networks; Optical fibers; Protection; Silicon; Space technology; Testing;
Conference_Titel :
Mass Storage Systems and Technologies, 2003. (MSST 2003). Proceedings. 20th IEEE/11th NASA Goddard Conference on
Print_ISBN :
0-7695-1914-8
DOI :
10.1109/MASS.2003.1194852
