Comparison of CMACs and radial basis functions for local function approximators in reinforcement learning

CMACs and radial basis functions are often used in reinforcement learning to learn value function approximations having local generalization properties. We examine the similarities and differences between CMACs, RBFs and normalized RBFs and compare the performance of Q-learning with each representation applied to the mountain car problem. We discuss ongoing research efforts to exploit the flexibility of adaptive units to better represent the local characteristics of the state space