The Design of Mass Storage System (MSS) Based on Hadoop

This paper studies the design of data storage system on Hadoop platform and has proposed the speculative data storage algorithm in view of data locality about power consumption modeling problem of MSS. Through calculating the time duration ratio of Map and Reduce tasks on each node, this algorithm introduces the system´s constituent parts consuming power in mathematical language and integrate them to obtain the overall power consumption model based on theoretical research with the combination of data locality on different nodes. Performance test of the Hadoop platform verifies efficiency and availability of this power consumption model via the comparison between estimated value with the model and measured value, and bring ideas for power consumption estimation of the MSS and related power consumption modeling.