Reinforcement learning generalization using state aggregation with a maze-solving problem

Reinforcement learning (RL) depends on constructing a lookup table for the value function of state-action pairs. Consequently, when learning in environments with large-scale state-action space, RL fails to achieve practical convergence rates. Therefore, the need for generalizing the original state-action space into more compact representation is crucial for many practical applications. In this paper, we propose a generalization technique using `state aggregation´. We apply this generalization technique to Q-learning, and show how to aggregate similar states together. The modified RL system architecture is presented along with the new algorithm. The proposed approach is tested and analyzed on a maze problem.