Adaptive Optimization of Time-out Policy for Dynamic Power Management Based on SMCP

Based on reinforcement learning, an adaptive online optimization algorithm of time-out policy is presented for dynamic power management. First the time-out policy driven power-managed systems are formulated as semi-Markov control processes. Under this analytic model, the equivalent effect on performance-power trade-off of time-out and stochastic policies is probed, and the equivalent relation between these two types policies is derived. Then an adaptive optimization algorithm that combines gradient estimation online and stochastic approximation is proposed. This algorithm doesn´t depend on the prior knowledge of system parameters, and can achieve a global optimum with less computational cost. Simulation results demonstrate the analytic results and the effectiveness of the proposed algorithm.