Reinforcement learning aided smart-home decision-making in an interactive smart grid

In this paper, a complete hierarchical architecture is presented for the Utility-customer interaction, which tightly connect several important research topics, such as customer load prediction, renewable generation integration, power-load balancing and demand response. The complete interaction cycle consists of two stages: (1) Initial interaction (long-term planning) and (2) Real-time interaction (short-term planning). A hidden mode Markov decision process (HM-MDP) model is developed for customer real-time decision making, which outperforms the conventional Markov decision process (MDP) model in handling the non-stationary environment. To obtain a low-complexity, real-time algorithm, that allows to adaptively incorporate new observations as the environment changes, we resort to Q-learning based approximate dynamic programming (ADP). Without requiring specific starting and ending points of the scheduling period, the Q-learning algorithm offers more flexibility in practice. Performance analysis of both exact and approximate algorithms are presented with simulation results, in comparison with other decision making strategies.