Open AccessJournal Article
Dynamic programming in constrained Markov decision processes
TL;DR: It is shown that the problem can be reformulated as a standard MDP and solved using the Dynamic Programming approach, and an example on arolled queue is presented.
read more
Abstract: We consider a discounted Markov Decision Process (MDP) supplemented with the requirement that another discounted loss must not exceed a specified value, almost surely. We show that the problem can be reformulated as a standard MDP and solved using the Dynamic Programming approach. An example on a con- trolled queue is presented. In the last section, we briefly reinforce the connection of the Dynamic Programming approach to another close problem statement and present the corresponding example. Several other types of constraints are discussed, as well.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
Stochastic optimal control with dynamic, time-consistent risk constraints
Yinlam Chow,Marco Pavone +1 more
- 17 Jun 2013
TL;DR: A dynamic programming approach to stochastic optimal control problems with dynamic, time-consistent risk constraints, which allows to compute the optimal costs by value iteration and a procedure to construct optimal policies.
Convex analytic approach to constrained discounted Markov decision processes with non-constant discount factors
TL;DR: In this article, the authors developed the convex analytic approach to a discounted discrete-time Markov decision process (DTMDP) in Borel state and action spaces with N constraints, and proved that every extreme point of the space of occupation measures can be generated by a deterministic stationary policy for the DTMDP.
17
Constrained discounted Markov decision processes with Borel state spaces
TL;DR: In this article, the authors studied discrete-time discounted constrained Markov decision processes (CMDPs) with Borel state and action spaces and provided general assumptions under which the optimization problems in CMDPs are solvable in the class of randomized stationary policies.
16
•Posted Content
A Martingale Approach and Time-Consistent Sampling-based Algorithms for Risk Management in Stochastic Optimal Control
TL;DR: In this article, a martingale approach is proposed to construct time-consistent control policies for stochastic optimal control problems with risk constraints that are expressed as bounded probabilities of failure for particular initial states.
14
•Posted Content
Stochastic Optimal Control With Dynamic, Time-Consistent Risk Constraints
Yinlam Chow,Marco Pavone +1 more
TL;DR: In this article, a dynamic programing approach to stochastic optimal control problems with dynamic, time-consistent risk constraints is presented, which allows to compute the optimal costs by value iteration.
12
References
•Book
Constrained Markov Decision Processes
Eitan Altman
- 30 Mar 1999
TL;DR: In this paper, a unified approach for the study of constrained Markov decision processes with a countable state space and unbounded costs is presented, where a single controller has several objectives; it is desirable to design a controller that minimize one of cost objectives, subject to inequality constraints on other cost objectives.
Dynamic programming equations for discounted constrained stochastic control
R. C. Chen,G.L. Blankenship +1 more
TL;DR: The application of the dynamic programming approach to constrained stochastic control problems with expected value constraints is demonstrated and optimality equations are obtained for these problems.
55
Constrained Markovian decision processes: the dynamic programming approach
Alexey Piunovskiy,Xuerong Mao +1 more
TL;DR: The main result is the constructive development of optimal strategy with the help of the dynamic programming method in semicontinuous controlled Markov models in discrete time with total expected losses.
46
On discounted dynamic programming with constraints
TL;DR: This paper introduces the Lagrangian programming problem corresponding to the original one, and proves the existence of an optimal solution for thelagrangian problem.
22
A variance‐constrained reservoir control problem
TL;DR: In this article, the variance constraint is incorporated as a penalty term in a nonseparable Lagrangian problem which is solved by a two-stage procedure, and the optimal solution to the non-separable problem is found by a simple search algorithm.
18