Q-learning in a competitive supply chain

The participants in a competitive supply chain take their decisions individually in a distributed environment and independent of one another. At the same time, they must coordinate their actions so that the total profitability of the supply chain is safeguarded. This decision problem is known to be a difficult one and the decisions at different stages of the supply chain may lead to large oscillations if not coordinated properly. In this paper, we consider reinforcement learning agents in a multi-echelon supply chain and study under which conditions they are able to manage the supply chain. Q-learning in the well-known beer game is used as a case. It is found that the reinforcement learning agents can learn better policies than humans, although they do not always converge to the optimal policy.