Incremental learning in dynamic environments using neural network with long-term memory

When the environment is dynamically changed for agents, knowledge acquired from an environment might be useless in the future environments. Therefore, agents should not only acquire new knowledge but also modify or delete old knowledge. However, these modification and deletion are not always efficient in learning. Because the knowledge once acquired in the past can be useful again in the future when the same environment reappears. To learn efficiently in this situation, agents should have memory to store old knowledge. In this paper, we propose an agent architecture that consists of four modules: resource allocating network (PAN), long-term memory (LTM), association buffer (A-Buffer), and environmental change detector (ECD). In LTM, not only acquired knowledge but also the information about which knowledge was produced in the same environment is stored. This information is utilized for recalling the knowledge acquired in the past when the same environment reappears. To evaluate the adaptability in a class of dynamic environments, we apply this model to a simple problem that some target functions to be approximated are changed in turn. As a result, we verify the following adaptability of RAN-ALTM: (1) incremental learning can be stably carried out, (2) environmental changes are correctly detected, (3) fast adaptation is realized by training some of the accumulated knowledge when the past environments reappear.