On the geometry of optimal control: the inverted pendulum example

We consider the structure of the solution of an interesting class of infinite horizon optimal control problems. We first verify that the local solution to such problems is very much like that of the linear-quadratic approximation, possessing a local C² value function and with optimal trajectories that are the image of trajectories on stable manifold of the Hamiltonian vector field. Then we proceed to study the nonlocal structure. This is accomplished by converting the infinite horizon problem into a finite horizon problem using the local C² value function as a terminal cost. Optimal trajectories of this finite horizon problem (and hence the infinite horizon problem) are shown to always exist and to be somewhat regular. Moreover, the optimal trajectories are always the image of Hamiltonian trajectories on the global stable manifold. This gives a result on the structure of the Hamiltonian global stable manifold. These ideas are illustrated using the familiar inverted pendulum on a cart example