Research of a heuristic reward function for reinforcement learning algorithms

The reward function is considered as the critical component for its effect of evaluating the action and guiding the reinforcement learning (RL) process. According to the distribution of rewards in the space of states, reward functions can have two basic forms, dense and sparse. We present an idea of designing a heuristic reward function in this paper. An additional reward is added to the traditional sparse reward function. The additional reward function F is a difference of potentials, which can provide more heuristic information for the learning system to progress rapidly. We also prove the convergence property of Q-value iteration. The heuristic reward function helps to implement an efficient reinforcement learning system on a real-time control or scheduling system.