Reinforcement learning algorithms for dynamic power management

In this paper we present a dynamic power management (DPM) framework based on model-free reinforcement learning (RL) techniques. For the RL algorithms, we employ both temporal difference learning and Q-learning for semi-Markov decision process in a continuous-time manner. The proposed DPM is model-free and do not require any prior information of the workload characteristics. The power manager learns the optimal power management policy that significantly reduces energy consumption while maintaining an acceptable performance level. Moreover, power-latency tradeoffs can be precisely controlled based on a user-defined parameter. In addition, the temporal difference (TD) learning is compared with the Q-learning approach in terms of both performance and convergence speed. Experiments on network cards show that TD achieves better power saving without sacrificing any latency and has faster convergence speed compared to Q-learning.