State thresholding to accelerate reinforcement learning

Along with the learning convergence and nonstationary equilibria, important thing to be solved in Reinforcement Learning is the slow-learning problem. In highly dynamic and stochastic systems, Markov Decision Processes is often used to model the situation, and RL is used to produce optimum control values which are expressed by optimum policy. However, approaches to solving MDP´s using RL depend on storing the optimal value function or Q-value function and action models as tables do not scale to large state-spaces. The number of states grow exponentially along with the number of agents, the state and action space, which cause the learning process become very slow since it needs a very large amount of computer memory, and yet it needs more computation than the most computer performance nowadays have offered. This paper addressed curse of dimensionality problem by reducing the states in the MDP iteratively using a novel algorithm called State Thresholding in Reinforcement Learning (STRL). STRL accelerate the learning process and empirically proven to outperformed Q learning algorithm.