Control of weakly-coupled Markov chains

A decomposition of weakly-coupled Markov chains into reduced-order aggregate chains and "fast" chains is derived. This decomposition is used to break an average cost per unit time problem into reduced-order subproblems, the solutions to which provide a near-optimal control. We consider the control of weakly-coupled Markov chains. We decompose this type of Markov chain into a reduced order aggregate "slow" chain together with a set of decoupled "fast" chains. This decomposition also separates the original cost function into cost functions associated with each subproblem, and leads to a set of individual control problems, with solutions providing a near-optimal control for the original problem. These results are illustrated through an example.