Reinforcement learning for sequential composition control

Sequential composition is an effective strategy for addressing complex control specifications and complex dynamical systems by partitioning the problem in time and space. Traditionally, sequential composition controllers are synthesized offline given a control task and a static environment with possible constraints. Dynamical environments may require redesigning the entire sequential composition controller, which may be time costly and inefficient. In this paper we introduce a learning strategy to augment online a pre-designed sequential composition controller based on reinforcement learning. By interpreting the sequential composition controller as an automaton, we add and delete nodes in the graph online, based on newly acquired knowledge via learning. We present simulation and experimental results for a nonlinear motion-control system.