Optimal Guaranteed Cost Tracking of Uncertain Nonlinear Systems Using Adaptive Dynamic Programming with Concurrent Learning

다국어 초록 (Multilingual Abstract)

In this paper, based on adaptive dynamic programming (ADP) with concurrent learning, the problem of optimal guaranteed cost tracking is studied for a class of uncertain continuous-time nonlinear systems. First, the original uncertain system and a reference system are combined into an augmented uncertain system, and the performance index is transformed into the corresponding augmented form. Afterwards, the solution of the optimal control problem consisting of the nominal augmented system with the modified performance index is proven to be the optimal guaranteed cost tracking control law of the original uncertain system. Moreover, a concurrent learning tuning algorithm based on ADP is presented to approximate the solution of corresponding Hamilton-Jacobi-Bellman (HJB) equation, which relaxes the condition of persistent excitation (PE). A neural network-based approximate optimal guaranteed cost tracking design is developed not only to ensure tracking error convergence to zero for alladmissible uncertainties but also to achieve the minimal guaranteed cost. Finally, two simulation examples are considered to verify the effectiveness of the theoretical results.

참고문헌 (Reference)

1 Q. Wei, "Value iteration adaptive dynamic programming for optimal control of discrete-time nonlinear systems" 46 (46): 840-853, 2016

2 P. Yu, "Robust tracking and disturbance rejection for linear uncertain system with unknown state delay and disturbance" 23 (23): 1445-1455, 2018

3 K. Tanaka, "Robust stabilization of a class of uncertain nonlinear systems via fuzzy control:quadratic stabilizability, H∞ control theory, and linear matrix inequalities" 4 (4): 1-13, 1996

4 K. Zhou, "Robust and Optimal Control" Prentice hall 1996

5 Y. Jiang, "Robust adaptive dynamic programming and feedback stabilization of nonlinear systems" 25 (25): 882-893, 2014

6 X. Yang, "Reinforcement learning for robust adaptive control of partially unknown nonlinear systems subject to unmatched uncertainties" 463 : 307-322, 2018

7 Derek Hoffman, "Quaternion-based Robust Trajectory Tracking Control of a Quadrotor Hover System" 제어·로봇·시스템학회 16 (16): 2575-2584, 2018

8 D. Liu, "Policy iteration adaptive dynamic programming algorithm for discrete-time nonlinear systems" 25 (25): 621-634, 2014

9 Lei Liu, "Partial State Constraints-Based Control for Nonlinear Systems With Backlash-Like Hysteresis" Institute of Electrical and Electronics Engineers (IEEE) 1-5, 2018

10 S. R. Moheimani, "Optimal guaranteed cost control of uncertain systems via static and dynamic output feedback" 32 (32): 575-579, 1996

11 Y. Huang, "Optimal guaranteed cost control of uncertain non-linear systems using adaptive dynamic programming with concurrent learning" 12 (12): 1025-1035, 2018

12 P. Shi, "Optimal guaranteed cost control of uncertain discrete time-delay systems" 157 (157): 435-451, 2003

13 L. Yu, "Optimal guaranteed cost control of discrete-time uncertain systems with both state and input delays" 338 (338): 101-110, 2001

14 I. R. Petersen, "Optimal guaranteed cost control and filtering for uncertain linear systems" 39 (39): 1971-1977, 1994

15 F. L. Lewis, "Optimal control" John Wiley & Sons 2012

16 B. Kiumarsi, "Optimal and autonomous control using reinforcement learning: A survey" 29 (29): 2042-2062, 2018

17 C. Qin, "Online optimal tracking control of continuous-time linear systems with unknown dynamics by using adaptive dynamic programming" 87 (87): 1000-1009, 2014

18 K. G. Vamvoudakis, "Online actor–critic algorithm to solve the continuous-time infinite horizon optimal control problem" 46 (46): 878-888, 2010

19 D. Liu, "Neural-network-based online HJB solution for optimal robust guaranteed cost control of continuous-time uncertain nonlinear systems" 44 (44): 2834-2847, 2014

20 C. Mu, "Neural-network-based adaptive guaranteed cost control of nonlinear dynamical systems with matched uncertainties" 245 : 46-54, 2017

21 L. Liu, "Neural networks-based adaptive finite-time fault-tolerant control for a class of strict-feedback switched nonlinear systems" 49 (49): 2536-2545, 2019

22 D. Wang, "Neural network robust tracking control with adaptive critic framework for uncertain nonlinear systems" 97 : 11-18, 2018

23 Bo Zhao, "Model-free Adaptive Dynamic Programming Based Near-optimal Decentralized Tracking Control of Reconfigurable Manipulators" 제어·로봇·시스템학회 16 (16): 478-490, 2018

24 H. Modares, "Linear quadratic tracking control of partially-unknown continuous-time systems using reinforcement learning" 59 (59): 3051-3056, 2014

25 W. Gao, "Learning-based adaptive optimal tracking control of strict-feedback nonlinear systems" 29 (29): 2614-2624, 2018

26 D. Wang, "Learning and guaranteed cost control with event-based adaptive critic implementation" 29 (29): 6004-6014, 2018

27 T. Peng, "Invariance and stability for bounded uncertain system" 10 (10): 679-690, 1972

28 B. Kiumarsi, "H∞ control of linear discrete-time systems : Off-policy reinforcement learning" 78 : 144-152, 2017

29 J. Wu, "Guaranteed cost nonlinear tracking control of a boilerturbine unit : an LMI approach" 41 (41): 889-895, 2010

30 X. Yang, "Guaranteed cost neural tracking control for a class of uncertain nonlinear systems using adaptive dynamic programming" 198 : 80-90, 2016

31 Q. Zhang, "Event-based robust control for uncertain nonlinear systems using adaptive dynamic programming" 29 (29): 37-50, 2018

32 Q. Wei, "Discrete-time local value iteration adaptive dynamic programming : Convergence analysis" 48 (48): 875-891, 2018

33 D. Ye, "Decentralized reliable guaranteed cost control for large-scale nonlinear systems using actor-critic network" 320 : 121-128, 2018

34 C. Li, "Data-based optimal control for weakly coupled nonlinear systems using policy iteration" 48 (48): 511-521, 2018

35 H. White, "Artificial Neural Networks : Approximation and Learning Theory" Blackwell Publishers, Inc. 1992

36 Q. Zhou, "Approximationbased adaptive tracking control for mimo nonlinear systems with input saturation" 45 (45): 2119-2128, 2015

37 X. Yang, "An adaptive critic approach to event-triggered robust control of nonlinear systems with unmatched uncertainties" 28 (28): 3501-3519, 2018

38 L. Yu, "An LMI approach to guaranteed cost control of linear uncertain time-delay systems" 35 (35): 1155-1159, 1999

39 D. P. Li, "Adaptive neural tracking control for an uncertain state constrained robotic manipulator with unknown time-varying delays" 48 (48): 2219-2228, 2018

40 T. Gao, "Adaptive neural network-based control for a class of nonlinear purefeedback systems with time-varying full state constraints" 5 (5): 923-933, 2018

41 S. Chang, "Adaptive guaranteed cost control of systems with uncertain parameters" 17 (17): 474-483, 1972

42 X. Cui, "Adaptive dynamic programming for H∞ tracking design of uncertain nonlinear systems with disturbances and input constraints" 31 (31): 1567-1583, 2017

43 W. Gao, "Adaptive dynamic programming and adaptive optimal output regulation of linear systems" 61 (61): 4164-4169, 2016