Approximate dynamic programming based on Gaussian process regression for the perimeter patrol optimization problem

A methodology is presented in this paper for stochastic optimal control of unmanned aerial vehicle performing the task of perimeter patrol. The optimal control problem is modeled as a Markov decision processes, and an approximate policy iteration algorithm is used for the cost-to-go function (value function) by introducing Gaussian process regression, resulting in improved quality of the decisions made while retaining computationally feasibility. The approximate dynamic programming (ADP) framework is developed to tackle the issues, in which situations standard dynamic programming algorithms become computationally too demanding. As a nonparametric ADP algorithm, the Gaussian processes that provide the combination of the prior and noise models presents a sub-solution in a lower dimensional space by exploiting kernel-based method. The numerical results that corroborate the effectiveness of the proposed methodology are also provided.