Open AccessProceedings Article
Probabilistic Differential Dynamic Programming
Yunpeng Pan,Evangelos A. Theodorou +1 more
- 08 Dec 2014
- Vol. 27, pp 1907-1915
TL;DR: Compared with the classical DDP and a state-of-the-art GP-based policy search method, PDDP offers a superior combination of data-efficiency, learning speed, and applicability.
read more
Abstract: We present a data-driven, probabilistic trajectory optimization framework for systems with unknown dynamics, called Probabilistic Differential Dynamic Programming (PDDP). PDDP takes into account uncertainty explicitly for dynamics models using Gaussian processes (GPs). Based on the second-order local approximation of the value function, PDDP performs Dynamic Programming around a nominal trajectory in Gaussian belief spaces. Different from typical gradient-based policy search methods, PDDP does not require a policy parameterization and learns a locally optimal, time-varying control policy. We demonstrate the effectiveness and efficiency of the proposed algorithm using two nontrivial tasks. Compared with the classical DDP and a state-of-the-art GP-based policy search method, PDDP offers a superior combination of data-efficiency, learning speed, and applicability.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
•Posted Content
Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Images
TL;DR: In this article, a deep generative model, belonging to the family of variational autoencoders, is used to generate image trajectories from a latent space in which the dynamics is constrained to be locally linear.
772
•Proceedings Article
Embed to control: a locally Linear Latent dynamics model for control from raw images
Manuel Watter,Jost Tobias Springenberg,Joschka Boedecker,Martin Riedmiller +3 more
- 07 Dec 2015
TL;DR: In this paper, a deep generative model, belonging to the family of variational autoencoders, is used to generate image trajectories from a latent space in which the dynamics is constrained to be locally linear.
Safe Learning of Quadrotor Dynamics Using Barrier Certificates
Li Wang,Evangelos A. Theodorou,Magnus Egerstedt +2 more
- 21 May 2018
TL;DR: This paper presents a data-driven approach based on Gaussian processes that learns models of quadrotors operating in partially unknown environments that expands the barrier certified safe region based on an adaptive sampling scheme.
191
One-shot learning of manipulation skills with online dynamics adaptation and neural network priors
Justin Fu,Sergey Levine,Pieter Abbeel +2 more
- 01 Oct 2016
TL;DR: In this paper, a model-based reinforcement learning algorithm that combines prior knowledge from previous tasks with online adaptation of the dynamics model is developed, which enables highly sample-efficient learning even in regimes where estimating the true dynamics is very difficult.
148
•Posted Content
From Pixels to Torques: Policy Learning with Deep Dynamical Models
TL;DR: In this paper, a deep dynamical model that uses deep auto-encoders to learn a low-dimensional embedding of images jointly with a predictive model in this lowdimensional feature space is proposed.
143
References
Gaussian Processes For Machine Learning
Tanja Hueber
- 01 Jan 2016
TL;DR: The gaussian processes for machine learning is universally compatible with any devices to read, and is available in the digital library an online access to it is set as public so you can get it instantly.
10K
Gaussian processes for machine learning
Carl E. Rasmussen,Christopher K. I. Williams +1 more
- 23 Nov 2005
TL;DR: The book provides a long-needed, systematic and unified treatment of theoretical and practical aspects of GPs in machine learning, targeted at researchers and students in machine learning and applied statistics.
4.6K
•Proceedings Article
Sparse Gaussian Processes using Pseudo-inputs
Edward Snelson,Zoubin Ghahramani +1 more
- 05 Dec 2005
TL;DR: It is shown that this new Gaussian process (GP) regression model can match full GP performance with small M, i.e. very sparse solutions, and it significantly outperforms other approaches in this regime.
•Proceedings Article
PILCO: A Model-Based and Data-Efficient Approach to Policy Search
Marc Peter Deisenroth,Carl Edward Rasmussen +1 more
- 28 Jun 2011
TL;DR: PILCO reduces model bias, one of the key problems of model-based reinforcement learning, in a principled way by learning a probabilistic dynamics model and explicitly incorporating model uncertainty into long-term planning.
Sparse on-line Gaussian processes
Lehel Csató,Manfred Opper +1 more
TL;DR: An approach for sparse representations of gaussian process (GP) models (which are Bayesian types of kernel machines) in order to overcome their limitations for large data sets is developed based on a combination of a Bayesian on-line algorithm and a sequential construction of a relevant subsample of data that fully specifies the prediction of the GP model.