Understand direct NDP with linear quadratic regulation

This work falls into the general area of approximate dynamic programming. Direct NDP designs are further analyzed using classic control-theoretic sensitivity arguments. The relationship between direct NDP and LQR designs are discussed due to their resemblances in system performance functions. The cart-pole benchmark problem is used to demonstrate the possibility that direct NDP sometimes produces LQR-like designs. It is also shown that given a well-designed direct NDP control system, one can find a corresponding LQR design which exhibits comparable closed-loop performance. The convergence properties are studied by taking direct NDP under the linear LQR control environment.