Open AccessJournal Article10.1016/J.INS.2012.07.006

An iterative adaptive dynamic programming algorithm for optimal control of unknown discrete-time nonlinear systems with constrained inputs

- 01 Jan 2013

- Vol. 220, pp 331-342

128

TL;DR: The adaptive dynamic programming approach is employed for designing an optimal controller of unknown discrete-time nonlinear systems with control constraints and a neural network is constructed for identifying the unknown dynamical system with stability proof.

About: This article is published in Information Sciences. The article was published on 01 Jan 2013. and is currently open access. The article focuses on the topics: Differential dynamic programming & Dual control theory.

AI Agents for this Paper

Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps

Most frequently asked questions

1. What are the contributions mentioned in the paper "Policy iteration adaptive dynamic programming algorithm for discrete-time nonlinear systems" ?

This paper is concerned with a new discrete-time policy iteration adaptive dynamic programming ( ADP ) method for solving the infinite horizon optimal control problem of nonlinear systems.. The main contribution of this paper is to analyze the convergence and stability properties of policy iteration method for discrete-time nonlinear systems for the first time.

2. What are the future works mentioned in the paper "Policy iteration adaptive dynamic programming algorithm for discrete-time nonlinear systems" ?

The convergence and stability properties of the discrete-time policy iteration algorithm with approximation errors will be discussed in their future work.

Fig. 9. Convergence trajectories of the first column of weights of the action network.

Fig. 4. Numerical results using value iteration algorithm. (a) Convergence trajectory of iterative performance index function. (b) Optimal trajectory of state x1. (c) Optimal trajectory of state x2. (d) Optimal control.

Fig. 1. Structure diagram of the algorithm.

Fig. 8. Convergence trajectories of the critic network.

Fig. 7. Numerical results of Example 3 using value iteration algorithm. (a) Convergence trajectory of iterative performance index function. (b) Iterative controls. (c) Iterative states. (d) Optimal states and control.

Fig. 5. Numerical results for nonquadratic utility function using policy iteration algorithm. (a) Convergence trajectory of iterative performance index function. (b) Iterative states. (c) Iterative controls. (d) Optimal states and control.

Citations

Journal Article•10.1109/TNNLS.2013.2281663

Policy Iteration Adaptive Dynamic Programming Algorithm for Discrete-Time Nonlinear Systems

Derong Liu, +1 more

- 01 Mar 2014

- IEEE Transactions on Neural Networks

TL;DR: It is shown that the iterative performance index function is nonincreasingly convergent to the optimal solution of the Hamilton-Jacobi-Bellman equation and it is proven that any of the iteratives control laws can stabilize the nonlinear systems.

...read moreread less

708

Journal Article•10.1109/TSMC.2020.3042876

Adaptive Dynamic Programming for Control: A Survey and Recent Advances

Derong Liu, +4 more

- 01 Jan 2021

- IEEE Transactions on Systems, Man, and C...

TL;DR: In this article, the adaptive dynamic programming (ADP) with applications in control is reviewed, and the use of ADP to solve game problems, mainly nonzero-sum game problems is elaborated.

...read moreread less

500

Journal Article•10.1109/TFUZZ.2015.2418000

Fuzzy Approximation-Based Adaptive Backstepping Optimal Control for a Class of Nonlinear Discrete-Time Systems With Dead-Zone

Yan-Jun Liu, +3 more

- 01 Feb 2016

- IEEE Transactions on Fuzzy Systems

TL;DR: An adaptive fuzzy optimal control design is addressed for a class of unknown nonlinear discrete-time systems that contain unknown functions and nonsymmetric dead-zone and can be proved based on the difference Lyapunov function method.

...read moreread less

409

Journal Article•10.1109/TCYB.2015.2417170

Reinforcement-Learning-Based Robust Controller Design for Continuous-Time Uncertain Nonlinear Systems Subject to Input Constraints

Derong Liu, +3 more

- 09 Apr 2015

- IEEE Transactions on Systems, Man, and C...

TL;DR: A novel RL-based robust adaptive control algorithm is developed for a class of continuous-time uncertain nonlinear systems subject to input constraints that is converted to the constrained optimal control problem with appropriately selecting value functions for the nominal system.

...read moreread less

395

•Journal Article•10.1109/TNNLS.2013.2280013

Decentralized Stabilization for a Class of Continuous-Time Nonlinear Interconnected Systems Using Online Learning Optimal Control Approach

Derong Liu, +2 more

- 01 Feb 2014

- IEEE Transactions on Neural Networks

TL;DR: It is proven that the decentralized control strategy of the overall system can be established by adding appropriate feedback gains to the optimal control policies of the isolated subsystems, and an online policy iteration algorithm is presented to solve the Hamilton-Jacobi-Bellman equations.

...read moreread less

340

...

Expand

References

Neuro-Dynamic Programming.

Dimitri P. Bertsekas

- 01 Jan 2009

TL;DR: In this article, the authors present the first textbook that fully explains the neuro-dynamic programming/reinforcement learning methodology, which is a recent breakthrough in the practical application of neural networks and dynamic programming to complex problems of planning, optimal decision making, and intelligent control.

...read moreread less

4.7K

Journal Article•10.1109/MCAS.2009.933854

Reinforcement learning and adaptive dynamic programming for feedback control

Frank L. Lewis, +1 more

- 01 Sep 2009

- IEEE Circuits and Systems Magazine

TL;DR: This work describes mathematical formulations for reinforcement learning and a practical implementation method known as adaptive dynamic programming that give insight into the design of controllers for man-made engineered systems that both learn and exhibit optimal behavior.

...read moreread less

1.5K

Journal Article•10.1016/J.AUTOMATICA.2010.02.018

Online actor-critic algorithm to solve the continuous-time infinite horizon optimal control problem

Kyriakos G. Vamvoudakis, +1 more

- 01 May 2010

- Automatica

TL;DR: An online algorithm based on policy iteration for learning the continuous-time optimal control solution with infinite horizon cost for nonlinear systems with known dynamics, which finds in real-time suitable approximations of both the optimal cost and the optimal control policy, while also guaranteeing closed-loop stability.

...read moreread less

1.3K

Journal Article•10.1016/J.AUTOMATICA.2004.11.034

Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach

Murad Abu-Khalaf, +1 more

- 01 May 2005

- Automatica

TL;DR: It is shown that the constrained optimal control law has the largest region of asymptotic stability (RAS) and the result is a nearly optimal constrained state feedback controller that has been tuned a priori off-line.

...read moreread less

1.3K

Journal Article•10.1109/72.623201

Adaptive critic designs

Danil V. Prokhorov, +1 more

- 01 Sep 1997

- IEEE Transactions on Neural Networks

TL;DR: In this paper, the authors discuss a variety of adaptive critic designs (ACDs) for neuro-control, which are suitable for learning in noisy, nonlinear, and nonstationary environments They have common roots as generalizations of dynamic programming for neural reinforcement learning approaches.

...read moreread less

1.1K

...

Expand

An iterative adaptive dynamic programming algorithm for optimal control of unknown discrete-time nonlinear systems with constrained inputs

Chat with Paper

AI Agents for this Paper

Most frequently asked questions

1. What are the contributions mentioned in the paper "Policy iteration adaptive dynamic programming algorithm for discrete-time nonlinear systems" ?

2. What are the future works mentioned in the paper "Policy iteration adaptive dynamic programming algorithm for discrete-time nonlinear systems" ?

Figures

Citations

Policy Iteration Adaptive Dynamic Programming Algorithm for Discrete-Time Nonlinear Systems

Adaptive Dynamic Programming for Control: A Survey and Recent Advances

Fuzzy Approximation-Based Adaptive Backstepping Optimal Control for a Class of Nonlinear Discrete-Time Systems With Dead-Zone

Reinforcement-Learning-Based Robust Controller Design for Continuous-Time Uncertain Nonlinear Systems Subject to Input Constraints

Decentralized Stabilization for a Class of Continuous-Time Nonlinear Interconnected Systems Using Online Learning Optimal Control Approach

References

Neuro-Dynamic Programming.

Reinforcement learning and adaptive dynamic programming for feedback control

Online actor-critic algorithm to solve the continuous-time infinite horizon optimal control problem

Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach

Adaptive critic designs

Related Papers (5)

Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach

Discrete-Time Nonlinear HJB Solution Using Approximate Dynamic Programming: Convergence Proof

Adaptive critic designs

Online actor-critic algorithm to solve the continuous-time infinite horizon optimal control problem

Online learning control by association and reinforcement