On the impact of coding parameters on storage requirement of region-based fault tolerant distributed file system design

Advances in technology have resulted in Internet-scale deployment of storage systems such as peer-to-peer storage and cloud storage, where data is distributed over multiple storage nodes in a networked environment. In these environments the storage nodes are often commodity machines and are susceptible to failure. The notion of fault domain, introduced by Microsoft Azure, captures the fault-tolerance aspects of a data center. A fault domain is defined as a set of servers all of which become inaccessible when a single fault (such as the failure of a switch or a router) occurs in the data center. As such a fault domain can be viewed as a spatially correlated or region based failure. In order to enhance reliability through redundancy, maximum distance separable (MDS) codes such as Reed-Solomon codes and (N, K) codings are utilized. In this paper we present analytical results demonstrating that the choice of the coding parameters N and K may have significant impact on storage that will be necessary to achieve reliability. We present a polynomial time algorithm for optimal storage allocation in a mesh network and we conduct extensive experimentation to evaluate the impact of the coding parameters N and K on the storage requirement to provide all region fault tolerance with varying size of the mesh and the fault region.