Application of actor-critic learning to adaptive state space construction

In order to adopt reinforcement learning for complicated and continuous systems, an adaptive control scheme based on normalized radial basis function under the structure of actor-critic is proposed. The state value function and action-state value function are approximated by the identical normalized radial basis function neural network. Taking into account the adaptivity and computational efficiency, input layer and hidden layer of NRBF network are shared by the actor and critic. The units of the hidden layer can be adaptively added and deleted according to task requirement during the learning process. This method was applied to the balance of an inverted pendulum. The simulation result in the paper evaluates the validity of the proposed algorithm.