Strategic generation bidding using a learning algorithm through updates of supply offer selection propensities

This paper discusses a novel bidding strategy of a generation company (genco) in an hourly day-ahead market. In the proposed method, a genco learns the returns of supply offers and adapts its strategy accordingly, based on the Variant Roth-Erev (VRE) reinforcement learning algorithm. Every supply offer submitted to the market receives a profit at the end of each day, and is strategically updated for the next day based on this profit. The novelty of our proposed method is that every supply offer has a propensity (an inclination or a tendency) to be selected associated with it. The propensity is updated as a percentage relative to every other supply offer´s propensity based on the profit received. The DC optimal power flow problem solved by the system operator is also improved by including the physical inter-temporal constraints such as the generator ramp rates, in addition to the supply offers. Simulations on a 5-bus test system show that a genco learns to strategically bid in the market using the relative percentage propensity update technique. As a result, without any market regulations, the locational marginal prices increased by 29% on average.