Hyper-cubic discretization for TD learning based on autonomous decentralized approach

Adaptive resolution of function approximator is known to be important when we apply reinforcement learning to unknown problems. We propose to apply successive division and integration scheme of function approximation to temporal difference learning based on local curvature. TD learning in continuous state-space is based on non-constant values function approximation, which requires the simplicity of function approximator representation. We define bases and local complexity of function approximator in the similar way to the autonomous decentralized function approximation, but they are much simpler. The simplicity of approximator element bring us much less computation and easier analysis. The proposed function approximator is proven to be effective through function approximation problem and a reinforcement learning standard problem, pendulum swing-up task.