Unified and efficient HEC storage system with a working-set based reorganization scheme

High-end computing (HEC) applications and simulations have become increasingly data intensive. The pressure on the storage system capability has substantially increased in recent years. Traditional hard disk drives (HDD) are dominant storage devices in HEC, but suffer seek time delays and rotational latencies. The emerged storage class memory such as Solid State Drive (SSD) provides a new promising high bandwidth and low latency storage solution, but with inherent limitations of small capacity, limited writing cycles, and high cost. SSDs and HDDs have complement characteristics in nature and there is a desire to combine and unify them to best serve HEC workloads. In this paper, we introduce our initial study of a novel Working-Set based Reorganization Scheme (WS-ROS), to manage and leverage the merits of both SSDs and HDDs and to provide a highly efficient storage system. The goal of the proposed scheme is to leverage SSDs service more read requests through keeping active data and to leverage HDDs to keep large non-active data and extend SSDs writing cycles. The preliminary evaluations have shown that the proposed scheme has significant advantages and holds a promise as a unified storage management scheme for HEC workloads.