Formalizing Multi-state Learning Dynamics

This paper extends the link between evolutionary game theory and multi-agent reinforcement learning to multistate games. In previous work, we introduced piecewise replicator dynamics, a combination of replicators and piecewise models to account for multi-state problems. We formalize this promising proof of concept and provide definitions for the notion of average reward games, pure equilibrium cells and finally, piecewise replicator dynamics. These definitions are general in the number of agents and states. Results show that piecewise replicator dynamics qualitatively approximate multi-agent reinforcement learning in stochastic games.