Application of actor-critic learning algorithm for optimal bidding problem of a Genco

The optimal bidding for Genco in a deregulated power market is an involved task. The problem is formulated in the framework of Markov decision process (MDP), a discrete stochastic optimization method. When the time span considered is 24 h, the temporal difference method becomes attractive for application. The cumulative profit over the span is the objective function to be optimized. The temporal difference technique and actor-critic learning algorithm are employed. An optimal strategy is devised to maximize the profit. The market-clearing system is included in the formulation. Simulation cases of three, seven, and ten participants are considered and the obtained results are discussed.