Using data-oriented storage method to build a high-parallel and high-efficiency disk cluster

In high energy physics and seismic physics, mass data is produced. Recent years, the ability of processor makes a rapid growth due to the development of IC technology, while the IO speed of data storage growth slowly, especially for the magnetic media based hard disks which is widely used for mass data storage. Traditional ways to increase the performance of hard disks show low efficiency due to the data couples of working disks. This paper presents a novel method of how to build a high-parallel and high-efficiency disk cluster. All the data is divided into different datasets, the datasets are stored into different disks based on a data-oriented method to cut off the relationship of them. Each dataset can be accessed independently, so much hard disks can work together efficiently. A prototype experiment result shows with the joint of 23 hard disks the cluster can do 3680 IO transfers per second (IOPS). Comparing with the 161.16 IOPS for single disk, the average IOPS of the cluster is 160.02, only 0.7% performance decreases due to the parallelization.