Applying temporal difference learning to acquire a high-performance position evaluation function

Chinese-Chess is more complex board game than International-Chess, it is usually played on a square grid containing 10 × 9 intersections. In Chinese-Chess Computer Game (CCCG), the most time-consuming aspect of building a high performance game playing program is the design, implementation and tuning of the position evaluation function. In this paper, a three-layer fully-connected feed forward neural network is designed as a position evaluation function. Temporal difference learning (TDL) is a reinforcement learning algorithm, which uses the difference of a pair of successive position-values to incrementally update the weights. Based on the three-layer neural network with single output, we derive a new weight updating rule for applying TD(λ) in CCCG. Starting with random initial weights between -0.5 and 0.5, the neural network is trained through the new rule on the grand-master database games. In the training process, each grand-master game is learned iteratively by the neural network until the evaluation value of the position in grand-master game becomes stable ultimately. In the experiments, we validate that our learned evaluation function is feasible and effective.