Proceedings Article10.1109/BRACIS.2014.34
Continuous Real Time Dynamic Programming for Discrete and Continuous State MDPs
Luis Gustavo Rocha Vianna,Scott Sanner,Leliane Nunes de Barros +2 more
- 18 Oct 2014
- pp 134-139
2
TL;DR: It is shown that using the initial state information greatly reduces the number of regions in the value function, therefore allowing CRTDP to solve DC-MDPs more efficiently than standard symbolic dynamic programming both in time and space required for the solution.
read more
Abstract: Applications of automated planning under uncertainty are often modelled as a discrete and continuous state Markov Decision Process (DC-MDP). Symbolic Dynamic Programming is the existent exact solution for DC-MDPs that uses the eXtended Algebraic Decision Diagrams (XADDs) to symbolically represent the state value function and that computes a complete state-space policy (which is very costly and limits solution to problems with small size and depth). Real-Time Dynamic Programming (RTDP) is an efficient solution method for discrete state MDPs that provides a partial solution for a known initial state. In this paper we combine the RTDP solution with XADD symbolic representation and computation of the value function to propose the Continuous Real Time Dynamic Programming (CRTDP) algorithm. This novel planner uses heuristic search and symbolic generalisation to efficiently update the value function by regions. We show that using the initial state information greatly reduces the number of regions in the value function, therefore allowing CRTDP to solve DC-MDPs more efficiently than standard symbolic dynamic programming both in time and space required for the solution.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
•Posted Content
Distributed Bayesian: a continuous Distributed Constraint Optimization Problem solver
TL;DR: This work theoretically shows that D-Bay converges to the global optimum of the DCOP for Lipschitz continuous utility functions and finds that the proposed algorithm generates better solutions while requiring less samples.
•Posted Content
An Anytime Algorithm for Reachability on Uncountable MDP.
Kush Grover,Jan Kretínský,Tobias Meggendorfer,Maximilian Weininger +3 more
- 10 Aug 2020
TL;DR: An algorithm for reachability on Markov decision processes with uncountable state and action spaces, which is the first such anytime algorithm, meaning that at any point in time it can return the current approximation with its precision.
References
•Book
Markov Decision Processes: Discrete Stochastic Dynamic Programming
Martin L. Puterman
- 15 Apr 1994
TL;DR: Puterman as discussed by the authors provides a uniquely up-to-date, unified, and rigorous treatment of the theoretical, computational, and applied research on Markov decision process models, focusing primarily on infinite horizon discrete time models and models with discrete time spaces while also examining models with arbitrary state spaces, finite horizon models, and continuous time discrete state models.
12.3K
Markov Decision Processes
P. Whittle,M. L. Puterman +1 more
TL;DR: Markov Decision Processes covers recent research advances in such areas as countable state space models with average reward criterion, constrained models, and models with risk sensitive optimality criteria, and explores several topics that have received little or no attention in other books.
Learning to act using real-time dynamic programming
TL;DR: An algorithm based on dynamic programming, which is called Real-Time DP, is introduced, by which an embedded system can improve its performance with experience and illuminate aspects of other DP-based reinforcement learning methods such as Watkins'' Q-Learning algorithm.
1.3K
Decision-theoretic planning: structural assumptions and computational leverage
TL;DR: In this article, the authors present an overview and synthesis of MDP-related methods, showing how they provide a unifying framework for modeling many classes of planning problems studied in AI.
Markov Decision Processes
Nicole Bäuerle,Ulrich Rieder +1 more
TL;DR: The theory of Markov Decision Processes is the theory of controlled Markov chains as mentioned in this paper, which has found applications in various areas like e.g. computer science, engineering, operations research, biology and economics.