Improving Speed and Efficiency of Dynamic Programming Methods through Chaos

A prominent weakness of the dynamic programming methods is that they perform operations throughout the entire set of states in a Markov decision process in every updating phase. In this paper, we propose a novel chaos-based method in order to solve the problem. For this purpose, a chaotic system is first initialized, and the resultant numbers are mapped onto the environment states through initial processing. In each traverse of the policy iteration method, policy evaluation is performed only once, and only a few states are updated. These states are proposed by the chaos system. In this method, the policy evaluation and improvement cycle lasts until an optimal policy is formulated in the environment. The same procedure is performed in the value iteration method, and only the values of a few states proposed by the chaos are updated in each traverse, whereas the values of the other states are left unchanged. Unlike the conventional methods, an optimal solution can be obtained in the proposed method by only updating a limited number of states that are properly distributed all over the environment by chaos. The test results indicate the improved speed and efficiency of the chaotic dynamic programming methods in obtaining the optimal solution in different grid environments.