Structure-Aware Stochastic Storage Management in Smart Grids

Demand-side management has been proposed as an important solution for improving the energy consumption efficiency in smart grids. However, traditional pricing-based demand-side management methods usually rely on the assumption that the statistics of the system dynamics (e.g., the time-varying electricity price, the arrival distribution of consumers´ demanded load) are known a priori, which does not hold in practice. In this paper, we propose a novel price-aware energy storage management algorithm for consumption scheduling which, unlike previous works, can operate optimally in systems where such statistical knowledge is unknown. We consider a power grid system where each consumer is equipped with an electrical energy storage device. Each consumer proactively determines how much energy to purchase from the energy producers by taking into consideration the time-varying and a priori unknown system dynamics, in order to maximize its own energy consumption utility. We first formulate the real-time energy storage management and demand response of the consumers as a Markov decision process and then propose an online learning algorithm that enables the consumers to learn the unknown system dynamics on-the-fly and have their energy storage management policies converge to the optimum. Our simulation results validate the efficacy of our algorithm, which helps consumers achieve higher average utility as opposed to other state-of-the-art online learning algorithms and energy storage management algorithms.