PRISCA: A policy search method for extreme trajectory following

Consider slide parking, given a desired demonstration, how to repeat it accurately? Many robotics tasks, such as slide parking, can be formulated in trajectory following, but not many dynamics of which can be easily modeled to facilitate a solving by the optimal control. Although an emerging stream in robotics is to learn the dynamics and policy from demonstrations, multiple, if not numerous, demonstrations are required. Therefore, learning a policy from scarce experience remains a difficult problem. In this paper, we proposed an online algorithm to learn a policy for control using only a desired demonstration and our intuitive knowledge of the dynamics system. Our approach is found on this observation: For trajectory following, even on a highly nonlinear and coupled dynamical system, so long as the state deviation is initially small, a policy can be updated online to keep the robot on track according to a very obvious and coarse model information (e.g., for driving, this information is simply: steer left to turn left). Our policy search is then devised as a function minimization problem, and is solved by gradient descent using the techniques of optimal baseline, least-state-deviation error, smoothing and in an inverse depreciation as a cost intensifier. Apart from guarantees on performance and convergence, we also demonstrated its performance in two simulations, and an extreme trajectory-following scenario - four-wheel-drive slide parking experiment. To our best knowledge, it is the state-of-the-art autonomous precision slide parking of a 4×4 brakeless RC car.