An Adaptive Approach for the Exploration-Exploitation Dilemma in Non-stationary Environment

A central problem in reinforcement learning is balancing exploration-exploitation dilemma in non-stationary environment. To address this problem, a data-driven Q-learning is presented. In this study, firstly, the information system of behavior is formed by experience of agent. Then the trigger mechanism of environment is constructed to trace changes of environment by uncertain knowledge of information system. The dynamic information of environment is used to balance exploration-exploitation dilemma with self-driven way. We illustrated this algorithm with grid-world navigation tasks. The results of simulated experiments show that this algorithm improves learning efficiency obviously.