DocumentCode :
786730
Title :
Reinforcement-Learning-Based Dual-Control Methodology for Complex Nonlinear Discrete-Time Systems With Application to Spark Engine EGR Operation
Author :
Shih, Peter ; Kaul, Brian C. ; Jagannathan, S. ; Drallmeier, James A.
Author_Institution :
Dept. of Electr. & Comput. Eng., Univ. of Sci. & Technol., Rolla, MO
Volume :
19
Issue :
8
fYear :
2008
Firstpage :
1369
Lastpage :
1388
Abstract :
A novel reinforcement-learning-based dual-control methodology adaptive neural network (NN) controller is developed to deliver a desired tracking performance for a class of complex feedback nonlinear discrete-time systems, which consists of a second-order nonlinear discrete-time system in nonstrict feedback form and an affine nonlinear discrete-time system, in the presence of bounded and unknown disturbances. For example, the exhaust gas recirculation (EGR) operation of a spark ignition (SI) engine is modeled by using such a complex nonlinear discrete-time system. A dual-controller approach is undertaken where primary adaptive critic NN controller is designed for the nonstrict feedback nonlinear discrete-time system whereas the secondary one for the affine nonlinear discrete-time system but the controllers together offer the desired performance. The primary adaptive critic NN controller includes an NN observer for estimating the states and output, an NN critic, and two action NNs for generating virtual control and actual control inputs for the nonstrict feedback nonlinear discrete-time system, whereas an additional critic NN and an action NN are included for the affine nonlinear discrete-time system by assuming the state availability. All NN weights adapt online towards minimization of a certain performance index, utilizing gradient-descent-based rule. Using Lyapunov theory, the uniformly ultimate boundedness (UUB) of the closed-loop tracking error, weight estimates, and observer estimates are shown. The adaptive critic NN controller performance is evaluated on an SI engine operating with high EGR levels where the controller objective is to reduce cyclic dispersion in heat release while minimizing fuel intake. Simulation and experimental results indicate that engine out emissions drop significantly at 20% EGR due to reduction in dispersion in heat release thus verifying the dual-control approach.
Keywords :
Lyapunov methods; closed loop systems; control system synthesis; discrete time systems; feedback; internal combustion engines; learning (artificial intelligence); neurocontrollers; nonlinear control systems; Lyapunov theory; adaptive neural network controller; closed-loop tracking error; complex feedback nonlinear discrete-time systems; complex nonlinear discrete-time systems; cyclic dispersion; dual-control methodology; exhaust gas recirculation; gradient-descent-based rule; nonstrict feedback; performance index; reinforcement-learning; spark engine EGR operation; spark ignition engine; state availability; uniformly ultimate boundedness; virtual control; Adaptive critic design; near-optimal control; nonstrict feedback nonlinear discrete-time system; output feedback control; separation principle; Algorithms; Artificial Intelligence; Automobiles; Computer Simulation; Energy Transfer; Gases; Hot Temperature; Models, Theoretical; Neural Networks (Computer); Nonlinear Dynamics; Pattern Recognition, Automated;
fLanguage :
English
Journal_Title :
Neural Networks, IEEE Transactions on
Publisher :
ieee
ISSN :
1045-9227
Type :
jour
DOI :
10.1109/TNN.2008.2000452
Filename :
4560233
Link To Document :
بازگشت