CRUSH: Data collection and analysis framework for power capped data intensive computing

Hadoop system has been widely used to support large-scale data intensive computing tasks on enterprise software infrastructure. Deploying and operating Hadoop incur high costs including hardware expenditure to build computer clusters and energy consumption to run the clusters. Building sustainable and scalable systems requires optimal power management schemes. In this paper, we present a framework, namely CRUSH, which collects power and resource usage data and provides statistical analysis for power management. We also demonstrate the usage of CRUSH in studying the effects of power capping on Hadoop jobs.