Proceedings Article10.48550/arXiv.2305.09930
Model-based Validation as Probabilistic Inference
Harrison Delecki,Anthony Joseph Corso,Mykel J. Kochenderfer +2 more
- 17 May 2023
pp 825-837
4
TL;DR: In this paper , the authors estimate the distribution over failure trajectories for sequential systems as Bayesian inference using rollouts of system dynamics and computes trajectory gradients using automatic differentiation.
read more
Abstract: Estimating the distribution over failures is a key step in validating autonomous systems. Existing approaches focus on finding failures for a small range of initial conditions or make restrictive assumptions about the properties of the system under test. We frame estimating the distribution over failure trajectories for sequential systems as Bayesian inference. Our model-based approach represents the distribution over failure trajectories using rollouts of system dynamics and computes trajectory gradients using automatic differentiation. Our approach is demonstrated in an inverted pendulum control system, an autonomous vehicle driving scenario, and a partially observable lunar lander. Sampling is performed using an off-the-shelf implementation of Hamiltonian Monte Carlo with multiple chains to capture multimodality and gradient smoothing for safe trajectories. In all experiments, we observed improvements in sample efficiency and parameter space coverage compared to black-box baseline approaches. This work is open sourced.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
Relative Policy-Transition Optimization for Fast Policy Transfer
TL;DR: A lemma based on existing theoretical results in reinforcement learning is introduced to measure the relativity between two arbitrary MDPs, that is the difference between any two cumulative expected returns defined on different policies and environment dynamics.
The Role of System Modeling on Artificial Intelligence: A Review of Emerging Trends
Saad Aldoihi,Khalid Alblalaihid,F. Alzemaia,Alia Almoajel +3 more
- 25 Oct 2023
TL;DR: The significance of system modeling in AI is explored, along with examples of its use in robotics, machine learning, and natural language processing, and the difficulties and possible directions in system modeling for AI are discussed.
RADIUM: Predicting and Repairing End-to-End Robot Failures using Gradient-Accelerated Sampling
Charles Dawson,Anjali Parashar,Chuchu Fan +2 more
TL;DR: Radium is a framework for predicting and repairing end-to-end robot failures using gradient-accelerated sampling. It efficiently handles high-dimensional environmental parameters, includes vision in the loop, and provides guidance on how to mitigate failures.
Learning-based Bayesian Inference for Testing of Autonomous Systems
Anjali Parashar,Ji Yin,Charles Dawson,Panagiotis Tsiotras,Chuchu Fan +4 more
TL;DR: This research introduces a novel sampling-based testing framework for autonomous systems, utilizing a discretized gradient-based second-order Langevin algorithm and learning-based techniques for constrained sampling of failure modes, improving failure prediction and feasibility.
References
•Posted Content
Proximal Policy Optimization Algorithms
TL;DR: A new family of policy gradient methods for reinforcement learning, which alternate between sampling data through interaction with the environment, and optimizing a "surrogate" objective function using stochastic gradient ascent, are proposed.
18K
Congested traffic states in empirical observations and microscopic simulations
TL;DR: It is shown that the results of the microscopic model can be understood by formulating the theoretical phase diagram for bottlenecks in a more general way, and a local drop of the road capacity induced by parameter variations has essentially the same effect as an on-ramp.
4.6K
MCMC using Hamiltonian dynamics
TL;DR: In this paper, the authors discuss theoretical and practical aspects of Hamiltonian Monte Carlo, and present some of its variations, including using windows of states for deciding on acceptance or rejection, computing trajectories using fast approximations, tempering during the course of a trajectory to handle isolated modes, and short-cut methods that prevent useless trajectories from taking much computation time.
•Posted Content
The No-U-Turn Sampler: Adaptively Setting Path Lengths in Hamiltonian Monte Carlo
Matthew D. Hoffman,Andrew Gelman +1 more
TL;DR: The No-U-Turn Sampler (NUTS) as discussed by the authors is an extension to HMC that eliminates the need to set a number of steps L. NUTS uses a recursive algorithm to build a set of likely candidate points that spans a wide swath of the target distribution, stopping automatically when it starts to double back and retrace its steps.
2.7K
Particle Markov chain Monte Carlo methods
TL;DR: It is shown here how it is possible to build efficient high dimensional proposal distributions by using sequential Monte Carlo methods, which allows not only to improve over standard Markov chain Monte Carlo schemes but also to make Bayesian inference feasible for a large class of statistical models where this was not previously so.
2.4K