Adaptive action selection using utility-based reinforcement learning

A basic problem of intelligent systems is choosing adaptive action to perform in a non-stationary environment. Due to the combinatorial complexity of actions, agent cannot possibly consider every option available to it at every instant in time. It needs to find good policies that dictate optimum actions to perform in each situation. This paper proposes an algorithm, called UQ-learning, to better solve action selection problem by using reinforcement learning and utility function. Reinforcement learning can provide the information of environment and utility function is used to balance exploration-exploitation dilemma. We implement our method with maze navigation tasks in a non-stationary environment. The results of simulated experiments show that utility-based reinforcement learning approach is more effective and efficient compared with Q-learning and recency-based exploration.