On the scalability of storage sub-system back-end networks

As the number of disks in a storage sub-system keeps increasing, the back-end network may become the bottleneck and thus impose a constraint on system performance. Unfortunately, there is no visible literature base on storage back-end network bottlenecks and scalability. This paper studies the scalability of the back-end network by answering two questions: (1) which factors affect the back-end network bandwidth requirements of disks? (2) given an interconnection network, how many disks can be connected to the system? Both analytical and simulation approaches are used. The major results are: (1) the back-end network bandwidth requirement per disk is primarily determined by workload features and the stripe unit size; (2) the number of disks that a system can accommodate is limited by the back-end network bandwidth. Systems with smaller stripe unit size have better network scalability. We present accurate quantitative results on the bandwidth requirement and the back-end network saturation points for a number of conventional system configurations under OLTP-like workloads and sequential workloads. We believe that these results provide practical guidance for storage sub-system design.