Asynchronous control for coupled Markov decision systems

This paper considers optimal control for a collection of separate Markov decision systems that operate asynchronously over their own state spaces. Decisions at each system affect: (i) the time spent in the current state, (ii) a vector of penalties incurred, and (iii) the next-state transition probabilities. An example is a network of smart devices that perform separate tasks but share a common wireless channel. The model can also be applied to data center scheduling and to various types of cyber-physical networks. The combined state space grows exponentially with the number of systems. However, a simple strategy is developed where each system makes separate decisions. Total complexity grows only linearly in the number of systems, and the resulting performance can be pushed arbitrarily close to optimal.