A Reinfrocement Learning Approach to Online Learning in Control

In search for a reinforcement learning technique with the simplicity of tabular techniques and capability of dealing with continuous states like in fuzzy systems, we introduce a new approach, which we have called Adaptive Pseudo Fuzzy technique. The idea is to partition the state space with some window functions similar to fuzzy membership functions and learn the output values by changing the rules’ consequents directly. This is a kind of local function approximation similar to RBFs and CMACs, however, unlike RBFs it does not shift the basis, and unlike CMACs it does not go through complex coarse coding. This method represents the input as a fuzzy system and learns the output as a look-up-table. We employed this technique in Sarsa(λ) to build a general online controller and examined it for a pole-balancing problem.