Reinforcement learning approaches for dispersion games

This paper proposes new reinforcement learning approaches for dispersion games in multi-agent systems. Multi-agent systems can establish orderly systems autonomously through interaction with autonomous agents. We expect to be able to build flexible and robust systems for the environmental changes by using multi-agent system approaches. However, it is difficult for designers to preliminarily embed appropriate behaviors to avoid conflict because complex dynamics emerges by interaction between many agents. We apply the proposed approaches to the narrow road problem that many agents go by each other in narrow roads, and verify the effectivity of the proposed method. In the narrow road problem, it is the optimal strategy that an agent selects going forward and the other agent selects giving way. However, it is difficult for agents to decide which strategy to select because they cannot predict other agents´ behaviors beforehand. We employ the Q-learning that can adjust discount rates by using reliability to solve the above mentioned problems. The Q-learning using reliability can differentiate into agents preferring to go forward and agents preferring to give way. We solve the conflict problems generated in multi-agent systems through autonomous functional differentiation of the many learning agents. In addition, we try to decrease the generation of the perceptual aliasing problems by improving the perceptual ability of agents. Through experimental results, we showed that agents differentiated into two type of agents, and acquired stable conflict avoidance behaviors with high probability than the conventional Q-learning approaches.