Barrier Lyapunov Function-Based Safe Reinforcement Learning Algorithm for Autonomous Vehicles with System Uncertainty

Yuxiang Zhang,Xiaoling Liang,Shuzhi Sam Ge,Bingzhao Gao,Tong Heng Lee 제어로봇시스템학회 2021 제어로봇시스템학회 국제학술대회 논문집 Vol.2021 No.10

Guaranteed safety and performance under various circumstances remain technically critical and practically challenging for the wide deployment of autonomous vehicles. For such safety-critical systems, it will certainly be a requirement that safe performance should be ensured even during the reinforcement learning period in the presence of system uncertainty. To address this issue, a Barrier Lyapunov Function-based safe reinforcement learning algorithm (BLF- SRL) is proposed here for the formulated nonlinear system in strict-feedback form. This approach appropriately arranges the Barrier Lyapunov Function item into the optimized backstepping control method to constrain the state-variables in the designed safety region during learning when unknown bounded system uncertainty exists. More specifically, the overall system control is optimized with the optimized backstepping technique under the framework of Actor-Critic, which optimizes the virtual control in every backstepping subsystem. Wherein, the optimal virtual control is decomposed into Barrier Lyapunov Function items; and also with an adaptive item to be learned with deep neural networks, which achieves safe exploration during the learning process. Eventually, the principle of Bellman optimality is satisfied through iteratively updating the independently approximated actor and critic to solve the Hamilton-Jacobi-Bellman equation in adaptive dynamic programming. More notably, the variance of control performance under uncertainty is also reduced with the proposed method. The effectiveness of the proposed method is verified with motion control problems for autonomous vehicles through appropriate comparison simulations.

Adaptive Neural Tracking Control of Full-state Constrained Nonstrict-feedback Time-delay Systems with Input Saturation

Xin Liu,Chuang Gao,Huanqing Wang,Libing Wu,Yonghui Yang 제어·로봇·시스템학회 2020 International Journal of Control, Automation, and Vol.18 No.8

In this study, an adaptive neural backstepping control scheme is proposed for a class of nonstrict-feedback time-delay systems with input saturation, full-state constraints and unknown disturbances. A structural property of radial basis function neural network is presented to deal with the design from the nonstrict-feedback formation. This method does not require the parameter separation technique and its assumption. With the help of the Lyapunov-Krasovskii functionals and Young’s inequalities, the effects of time delays are compensated, and the unknown disturbances are eliminated in the design process. The barrier Lyapunov function (BLF) is applied to arrest the violation of the full-state constraints. To overcome the problem of input saturation nonlinearity, the smooth nonaffine function of the control input signal is adopted to approach the input saturation function. Moreover, an adaptive backstepping neural control strategy is proposed. The proposed adaptive neural controller ensures that all the closed-loop signals are semi-globally uniformly ultimately bounded (SGUUB). Furthermore, the tracking error can converge to a small neighborhood of the origin. The simulation result shows the effectiveness of this method.

Integral Barrier Lyapunov Functions-based Neural Control for Strictfeedback Nonlinear Systems with Multi-constraint

Jun Zhang 제어·로봇·시스템학회 2018 International Journal of Control, Automation, and Vol.16 No.4

A new robust tracking control approach is proposed for strict-feedback nonlinear systems with state and input constraints. The constraints are tackled by extending the control input as an extended state and introducing an integral barrier Lyapunov function (IBLF) to each step in a backstepping procedure. This extends current research on barrier Lyapunov functions(BLFs)-based control for nonlinear systems with state constraints to IBLF-based control for strict-feedback nonlinear systems with state and input constraints. Since the IBLF allows the original constraints to be mixed with the error terms, the use of IBLF decreases conservatism in barrier Lyapunov functionsbased control. In the backstepping procedure, neural networks (NNs) with projection modifications are applied to estimate system uncertainties, due to their ability in guaranteeing estimators in a given bounded area. To facilitate the use of the once-differentiable NNs estimators in the backstepping procedure, the virtual controllers are passed through command filters. Finally, simulation results are presented to illustrate the feasibility and effectiveness of the proposed control.

Barrier Lyapunov function-based model-free constraint position control for mechanical systems

한성익,하현욱,이장명 대한기계학회 2016 JOURNAL OF MECHANICAL SCIENCE AND TECHNOLOGY Vol.30 No.7

In this article, a motion constraint control scheme is presented for mechanical systems without a modeling process by introducing a barrier Lyapunov function technique and adaptive estimation laws. The transformed error and filtered error surfaces are defined to constrain the motion tracking error in the prescribed boundary layers. Unknown parameters of mechanical systems are estimated using adaptive laws derived from the Lyapunov function. Then, robust control used the conventional sliding mode control, which give rise to excessive chattering, is changed to finite time-based control to alleviate undesirable chattering in the control action and to ensure finite-time error convergence. Finally, the constraint controller from the barrier Lyapunov function is designed and applied to the constraint of the position tracking error of the mechanical system. Two experimental examples for the XY table and articulated manipulator are shown to evaluate the proposed control scheme.

Barrier Lyapunov Functions-based Output Feedback Control for a Class of Nonlinear Cascade Systems With Time-varying Output Constraints

Jing Yang,Jie Zhang,Zhongcai Zhang,Yuqiang Wu 제어·로봇·시스템학회 2024 International Journal of Control, Automation, and Vol.22 No.2

In this study, a class of nonlinear systems with integral input to state stability (iISS) inverse dynamics and unknown control direction are examined for the issue of time-varying asymmetric output constraints of adaptive output feedback controller. To deal with unmeasured state variables and unknown directions, the state observer is constructed using a Rickati matrix differential equation with time variation. A backstepping-based method is recommended for establishing the dynamic output feedback control law. By ensuring boundedness for the timedependent barrier Lyapunov function (BLF) in the closed loop, we may not only maintain the boundedness and stability of other signals, but also avoid breaking the time-varying asymmetric constraint of the output. Finally, simulation analyses are used to confirm the scheme’s efficacy.

Observer Based Nonlinear Control using Barrier Lyapunov Function for Position Tracking of Sawyer Motor under Yaw Constraint

Donghoon Shin,Youngwoo Lee,Wonhee Kim,Chung Choo Chung 제어로봇시스템학회 2016 제어로봇시스템학회 국제학술대회 논문집 Vol.2016 No.10

In this paper, we present an observer based nonlinear controller using barrier Lyapunov function (BLF) for position tracking of Sawyer motor under yaw constraint. The dynamics of Sawyer motor is modified to apply BLF-based backstepping control design. To relax the matching condition, torque and force modulations are proposed. The modulations enable decoupling forces and torque of the Sawyer motor, and facilitate implementing the design of backstepping controller with the BLF. In addition, we propose a state augmented nonlinear observer to estimate the velocities, yaw rate, load forces and load torque. We show the closed-loop stability with a composite Lyapunov function. Simulation results validate the effectiveness of the proposed method.

Neuro-adaptive Event-triggered Optimal Control for Power Battery Systems With State Constraints

Xiaoxuan Pei,Kewen Li,Yongming Li 제어·로봇·시스템학회 2024 International Journal of Control, Automation, and Vol.22 No.2

This paper investigates an adaptive neural networks (NNs) event-triggered optimal control method for the second-order resistance capacitance (RC) equivalent circuit system with state constraints. The NNs are used to estimate the unknown nonlinear functions. In order to constrain the states within the designed boundary in optimal control strategy, the barrier Lyapunov function (BLF) method is taken into account. Furthermore, to economic the transmission resources, the adaptive NNs event-triggered optimizing control strategy is developed by employing the relative threshold strategy. The proposed optimal control strategy is not only able to satisfy the stability of closedloop system, but also can guarantee the performance index functions minimized when all states remain within the given boundaries. Finally, the effectiveness of the suggested control method is demonstrated by simulation.

Finite-time Stabilization with Output-constraints of A Class of Highorder Nonlinear Systems

Ruicheng Ma,Bin Jiang,Yan Liu 제어·로봇·시스템학회 2018 International Journal of Control, Automation, and Vol.16 No.3

In this paper, the finite-time stabilization with output-constraint is investigated for a class of high-order nonlinear systems with the powers of positive odd rational numbers by constructing a Barrier Lyapunov function. First, sufficient conditions on characterizing the nonlinear functions of the considered systems are derived. Then, based on the technique of adding one power integrator, the global finite-time stabilizers of individual subsystems are systematically constructed to guarantee global finite-time stability with output constraints of the closed-loop nonlinear system. Finally, an example is provided to demonstrate the effectiveness of the proposed result.

Adaptive Fuzzy Backstepping Dynamic Surface Control for Output-constrained Non-smooth Nonlinear Dynamic System

한성익,이장명 제어·로봇·시스템학회 2012 International Journal of Control, Automation, and Vol.10 No.4

Output-constrained backstepping dynamic surface control (DSC) is proposed for the purpose of output constraint and precise output positioning of a strict feedback single-input, single-output dynamic system in the presence of deadzone and uncertainty. A symmetric barrier Lyapunov function (BLF) is employed to meet the output constraint requirement using DSC as an alternative method of backstepping control that is adopted mainly to deal with the BLF’s constraint control. However, using the ordinary DSC method with the BLF limits the selection of the control gain whereas this limitation does not exist in the backstepping structure. To remove this limitation, we propose a partial backstep-ping DSC method in which backstepping control is added only in the first recursive DSC design step. For precise positioning, an inverse deadzone method and adaptive fuzzy system are introduced to handle unknown deadzone and unmodeled nonlinear functions. We show that the semiglobal boundedness of the overall closed-loop signals is guaranteed, the tracking error converges within the prescribed region, and precise positioning performance is ensured. The proposed control scheme is experimentally evaluated using a robot manipulator.

Prescribed Performance-tangent Barrier Lyapunov Function for Adaptive Neural Backstepping Control of Variable Stiffness Actuator with Input and Output Constraints

Yu Xia,Jun-Yang Li,Yan-Kui Song,Jia-Xu Wang,Yan-Feng Han,Ke Xiao 제어·로봇·시스템학회 2023 International Journal of Control, Automation, and Vol.21 No.3

Due to the complexity of modeling and the strong transmission coupling, the rich background of rigid actuator control has not been transferred to variable stiffness actuator (VSA). Therefore, most model-based control techniques developed for VSA require feedback linearization first. Alternatively, VSA can use non-model-based control techniques such as PD control, but it does not show strong robustness under disturbances. This paper is concerned with designing a novel adaptive neural network backstepping control scheme without using feedback linearization for a special VSA with saturation inputs, output constraints, and disturbances. Firstly, for ensuring the VSA with lower tracking error and higher security, the prescribed performance-tangent barrier Lyapunov function (PP-TBLF) is introduced to handle the prescribed output performance constraints. Subsequently, the Chebyshev neural network and the Nussbaum-type function are exploited to approximate the unknown nonlinearities and unknown gains. Meanwhile, the inverse hyperbolic sine function tracking differentiator is utilized to solve the “explosion of complexity” caused by the differentiation of virtual inputs and also approximate the complex partial derivatives caused by the auxiliary control signals. Finally, the stability of the whole scheme is proved by the Lyapunov criterion. The simulation results illustrate the raised control scheme’s feasibility and show a better closed-loop behavior relative to that obtained using a classic PD controller.