A reinforcement learning algorithm for neural networks with incremental learning ability

When neural networks are used for approximating action-values of Reinforcement Learning (RL) agents, the "interference" caused by incremental learning can be serious. To solve this problem, in this paper, a neural network model with incremental learning ability was applied to RL problems. In this model, correctly acquired input-output relations are stored into long-term memory, and the memorized data are effectively recalled in order to suppress the interference. In order to evaluate the incremental learning ability, the proposed model was applied to two problems: Extended Random-Walk Task and Extended Mountain-Car Task. In these tasks, the working space of agents is extended as the learning proceeds. In the simulations, we certified that the proposed model could acquire proper action-values as compared with the following three approaches to the approximation of action-value functions: tile coding, a conventional neural network model and the previously proposed neural network model.