Collaboration of discrete-time Markov chains: Tensor and product form

We consider first discrete-time Markov chains (DTMCs) in competition over a set of resources. We build a multidimensional Markov chain based on the Cartesian product of the state space and on competition rules between the chains. We then generalize this approach to DTMCs with collaboration between components. The competition and the collaboration between chains simply assume that when a resource is owned by a component (or when a component is in a specific subset of states) it affects the transition probabilities of the other components of the chain. We prove that under some competition rules the steady-state distribution of the multidimensional chain has a product form. This work extends Boucherie’s theory based on continuous-time chains. The proof relies on algebraic properties of the generalized tensor product defined by Plateau and Stewart.