Solving the curse of dimensionality utilizing action-dependent heuristic dynamic programming

Dynamic programming is an effective optimal control method for multi-stage decision-making processes. However, it can´t be used to solve complex issues due to the problem of curse of dimensionality. Through analyzing the problem of dynamic programming, this article elaborates the theory and method of approximate dynamic programming solving this problem in detail. The second-order training algorithm is also given to improve the convergence performance of iteration and stability performance of training. Finally, this method was applied in the speed fluctuation control at idle for a four-cylinder diesel engine to verify its correctness and validity. Though illustrated for engine, this control system framework should also be applicable to general purpose nonlinear system, and it doesn´t need the model of the controlled object.