Multi mobile robot navigation using distributed value function reinforcement learning

In this paper we propose a new fuzzy-based navigation system for two intelligent mobile robots using distributed value function reinforcement learning. The robots use their sensors to provide information about their workspace. A fuzzy controller uses this information to select a proper action for the currently sensed state. The parameters of the input and output fuzzy membership functions are determined by a learning automation at each time step based on the sparseness of the obstacles. Therefore the robots learn to control their velocity, and attention range regarding the density of the obstacles in the workspace. The distributed approach enables the robots to learn more than one simple behavior concurrently. So, in contrast to the existing methods no behavior blending is needed. This approach also enables the robots to learn a value function, which is an estimate of future rewards for both of them. In other words cooperation is maintained and each robot learns to execute the actions that are good for both of them. The proposed controller has a very simple architecture and clear logic. The time and computation cost are low and it can adapt well to environment changes. Computer simulations are used to investigate the effectiveness of the controller.