Title :
Online Reinforcement Learning Neural Network Controller Design for Nanomanipulation
Author :
Yang, Qinmin ; Jagannathan, S.
Author_Institution :
Dept. of Electr. & Comput. Eng., Missouri-Rolla Univ., Rolla, MO
Abstract :
In this paper, a novel reinforcement learning neural network (NN)-based controller, referred to adaptive critic controller, is proposed for affine nonlinear discrete-time systems with applications to nanomanipulation. In the online NN reinforcement learning method, one NN is designated as the critic NN, which approximates the long-term cost function by assuming that the states of the nonlinear systems is available for measurement. An action NN is employed to derive an optimal control signal to track a desired system trajectory while minimizing the cost function. Online updating weight tuning schemes for these two NNs are also derived. By using the Lyapunov approach, the uniformly ultimate boundedness (UUB) of the tracking error and weight estimates is shown. Nanomanipulation implies manipulating objects with nanometer size. It takes several hours to perform a simple task in the nanoscale world. To accomplish the task automatically the proposed online learning control design is evaluated for the task of nanomanipulation and verified in the simulation environment
Keywords :
Lyapunov methods; discrete time systems; learning (artificial intelligence); neurocontrollers; nonlinear control systems; optimal control; Lyapunov approach; adaptive critic controller; affine nonlinear discrete-time systems; dynamic programming; long-term cost function; nanomanipulation; object manipulation; online reinforcement learning neural network controller; online updating weight tuning; optimal control signal; simulation environment; system trajectory; tracking error; uniformly ultimate boundedness; weight estimates; Adaptive control; Control systems; Cost function; Learning; Neural networks; Nonlinear control systems; Nonlinear systems; Optimal control; Programmable control; Trajectory; Lyapunov method; Neural network; dynamic programming; nanomanipulation; on-line learning; reinforcement learning;
Conference_Titel :
Approximate Dynamic Programming and Reinforcement Learning, 2007. ADPRL 2007. IEEE International Symposium on
Conference_Location :
Honolulu, HI
Print_ISBN :
1-4244-0706-0
DOI :
10.1109/ADPRL.2007.368192