Proceedings Article10.1109/CDC.1992.371410
Convex stochastic control problems
E. Fernandez-Gaucherand,Ari Arapostathis,S.I. Marcus +2 more
- 16 Dec 1992
- pp 2179-2180
15
TL;DR: In this paper, the authors focus on structured solutions to stochastic control models, which are models for which value functions and/or optimal policies have some special dependence on the (initial) state.
read more
Abstract: The solution of the infinite horizon stochastic control problem under certain criteria, the functional characterization and computation of optimal values and policies, is related to two dynamic programming-like functional equations: the discounted cost optimality equation (DCOE) and the average cost optimality equation (ACOE). The authors consider what useful properties, shared by large and important problem classes, can be used to show that an ACOE holds, and how these properties can be exploited to aid in the development of tractable algorithmic solutions. They address this issue by concentrating on structured solutions to stochastic control models. By a structured solution is meant a model for which value functions and/or optimal policies have some special dependence on the (initial) state. The focus is on convexity properties of the value function. >
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
Optimality Inequalities for Average Cost Markov Decision Processes and the Stochastic Cash Balance Problem
Eugene A. Feinberg,Mark E. Lewis +1 more
TL;DR: This work describes structural properties of average cost optimal policies for the cash balance problem; an inventory control problem where the demand may be negative and the decision-maker can produce or scrap inventory.
Infinite Horizon Strategies for Replenishment Systems with a General Pool of Suppliers
Awi Federgruen,Nan Yang +1 more
TL;DR: A general infinite horizon inventory control model that combines demand and supply risks and the firm's ability to mitigate the supply risks by diversifying its procurement orders among a set of N potential suppliers is considered.
38
Optimality of Four-Threshold Policies in Inventory Systems with Customer Returns and Borrowing/Storage Options
Eugene A. Feinberg,Mark E. Lewis +1 more
TL;DR: In this paper, the authors consider a single-commodity inventory system with negative demand, where the demand is modeled by a sequence of independent and identically distributed random variables that can take negative values.
Asymptotic Optimality of Constant-Order Policies in Joint Pricing and Inventory Control Models
TL;DR: It is proved that the best constant-order dynamic pricing policy is asymptotically optimal as the lead time grows large, which is exactly the setting in which the problem becomes computationally intractable due to the curse of dimensionality.
18
Value iteration in average cost Markov control processes on Borel spaces
TL;DR: In this article, the convergence of the value iteration algorithm to the optimal average cost was studied for discrete-time Markov control processes with Borel state and control spaces, with possiblyunbounded costs and noncompact control constraint sets, and the average cost criterion.
17
References
•Book
Stochastic optimal control : the discrete time case
Dimitri P. Bertsekas,Steven E. Shreve +1 more
- 01 Feb 2007
TL;DR: This research monograph is the authoritative and comprehensive treatment of the mathematical foundations of stochastic optimal control of discrete-time systems, including thetreatment of the intricate measure-theoretic issues.
2.1K
•Book
Dynamic Programming: Deterministic and Stochastic Models
Dimitri P. Bertsekas
- 01 Apr 1987
TL;DR: As one of the part of book categories, dynamic programming deterministic and stochastic models always becomes the most wanted book.
1.5K
•Book
Stochastic systems : estimation, identification, and adaptive control
P. R. Kumar,Pravin Varaiya +1 more
- 01 Jan 1986
TL;DR: The mathematics of filtering and ee/ise 556: stochastic systems fall 2013 usc search identification and system parameter estimation 1991 gbv is described.
1.2K
Discrete-time controlled Markov processes with average cost criterion: a survey
Aristotle Arapostathis,Vivek S. Borkar,E. Fernandez-Gaucherand,Mrinal K. Ghosh,Steven I. Marcus +4 more
TL;DR: A survey of the average cost control problem for discrete-time Markov processes can be found in this paper, where the authors have attempted to put together a comprehensive account of the considerable research on this problem over the past three decades.
Average Cost Optimal Stationary Policies in Infinite State Markov Decision Processes with Unbounded Costs
TL;DR: In this paper, the authors considered infinite state Markov decision processes with unbounded costs and provided sufficient conditions for the existence of a distinguished state of smallest discounted value and a single stationary policy inducing an irreducible, ergodic Markov chain for which the average cost of a first passage from any state to the distinguished state is finite.
237