Swarm reinforcement learning methods improving certainty of learning for a multi-robot formation problem

In this paper, we treat a multi-robot formation problem. In this problem, multiple robots move from their respective initial positions, and they achieve to make a given target formation by reaching goal positions. The goal positions which the robots reach must be different from each other. They learn their respective goal positions and the shortest routes to the goal positions. For solving the formation problem, we recently proposed a swarm reinforcement learning method. In the method, multiple sets of the robots and environments, which are called learning worlds, are prepared and the robots in each learning world learn not only by performing a usual reinforcement learning method but also by exchanging information among the learning worlds. The method, however, sometimes fails to find not only an optimal policy but also a policy which enables the robots to make the target formation, especially in the case where the target formation is large and complicated. In order to resolve this problem, this paper proposes swarm reinforcement learning methods in which the robots learn through always taking actions according to the current policy in learning by the usual reinforcement learning method. The performance of the proposed methods is evaluated through simulated experiments.