Cooperative Q-learning with heterogeneity in actions

Cooperation in learning improves the speed of convergence and the quality of learning. Special care is needed when heterogeneous agents cooperate in learning. It is discussed that, cooperation in learning may cause the learning process to diverge if heterogeneity is not handled properly. In this paper, it is assumed that two heterogeneous Q-learning agents cooperate to learn. The heterogeneity is assumed in their action order (and not in their action set). A Q-learning-based method is introduced for the agents to learn the mapping among their actions. It is shown that, the agents are able to learn this mapping while cooperating in learning. Some simulation results are reported to show the effectiveness of the proposed method.