An hybrid methodology for RL-based behavior coordination in a target following mission with an AUV

Proposes a behavior-based scheme for high-level control of autonomous underwater vehicles (AUVs). Two main characteristics can be highlighted in the control scheme. Behavior coordination is done through a hybrid methodology, which takes in advantages of the robustness and modularity in competitive approaches, as well as optimized trajectories in cooperative ones. As a second feature, behavior state/action mapping is learnt by means of reinforcement learning (RL). A continuous Q-learning algorithm, implemented with a feed-forward neural network, is used. The behavior-based scheme attempts to fulfill simple missions in which several behaviors/tasks compete for the vehicle´s control. The paper shows its feasibility with a target following mission designed to be carried out in a pool with the AUV ODIN. In the paper, simulation results are shown demonstrating the good performance of the hybrid method on behavior coordination as well as the convergence of the RL-based behaviors