A reinforcement learning algorithm for Agent-based Computational Economics (ACE) model of electricity markets

Electricity markets in countries around the world are being restructured in pursuit of economic efficiency through competition. However, unpredictability in electricity prices and previous occurrences of market failure have indicated a need to better understand the complex interactions between the various market participants as well as to design market rules that maximizes efficiency and security. In this paper, the techniques of Agent-based Computational Economics (ACE) are employed to simulate the behavior of the GenCo participants in the National Electricity Market of Singapore (NEMS). The use of Reinforcement Learning (RL) in the agent-based modeling will be a more realistic representation of the GenCo and will bring about more accurate electricity market simulation outcomes. Actor-Critics constitute an important aspect of RL and in this paper we propose an adaptive actor-critic mapping using Particle Swarm Optimization (PSO). A simulation platform is built with the proposed model for Genco´s learning and is tested in the conditions of varying vesting contract levels. Simulation results indicate that the proposed learning algorithm is able to procure higher GenCo revenue when benchmarked with existing learning algorithms.