/pdf/optimising-pid-control-with-residual-policy-reinforcement-205of43214.pdf

Optimising PID Control with Residual Policy Reinforcement Learning.

Deep learning is emerging as a multilayer neural network dimensionality reduction algorithm, the neural network model with multiple deep hidden layer through the formation of the high dimensional data of input layer feature extraction, to find a low dimensional nested data structure, the formation of more abstract effective executives said. Starting from the basic principle of deep learning algorithm, introduces the basic structure of single-layer network restricted Boltzmann deep learning algorithm and its training process. Finally, an example is given to illustrate the performance improvement of deep learning technology applied to handwritten digit recognition, and the deep learning technology is briefly summarized.

/pdf/the-principle-and-application-of-deep-learning-algorithm-5fndzzsmfz.pdf

The Principle and Application of Deep Learning Algorithm

In this survey, we design formal verification and control algorithms for autonomous systems with practical safety guarantees using conformal prediction (CP), a statistical tool for uncertainty quantification. We focus on learning-enabled autonomous systems (LEASs) in which the complexity of learning-enabled components (LECs) is a major bottleneck that hampers the use of existing model-based verification and design techniques. Instead, we advocate for the use of CP, and we will demonstrate its use in formal verification, systems and control theory, and robotics. We argue that CP is specifically useful due to its simplicity (easy to understand, use, and modify), generality (requires no assumptions on learned models and data distributions, i.e., is distribution-free), and efficiency (real-time capable and accurate). We pursue the following goals with this survey. First, we provide an accessible introduction to CP for non-experts who are interested in using CP to solve problems in autonomy. Second, we show how to use CP for the verification of LECs, e.g., for verifying input-output properties of neural networks. Third and fourth, we review recent articles that use CP for safe control design as well as offline and online verification of LEASs. We summarize their ideas in a unifying framework that can deal with the complexity of LEASs in a computationally efficient manner. In our exposition, we consider simple system specifications, e.g., robot navigation tasks, as well as complex specifications formulated in temporal logic formalisms. Throughout our survey, we compare to other statistical techniques (e.g., scenario optimization, PAC-Bayes theory, etc.) and how these techniques have been used in verification and control. Lastly, we point the reader to open problems and future research directions. 

Formal Verification and Control with Conformal Prediction

In this paper, we consider the problem of synthesizing a controller in the presence of uncertainty such that the resulting closed-loop system satisfies certain hard constraints while optimizing certain (soft) performance objectives. We assume that the hard constraints encoding safety or mission-critical specifications are expressed using Signal Temporal Logic (STL), while performance is quantified using standard cost functions on system trajectories. To ensure satisfaction of the STL constraints, we algorithmically obtain control barrier functions (CBFs) from the STL specifications. We model controllers as neural networks (NNs) and provide an algorithm to train the NN parameters to simultaneously optimize the performance objectives while satisfying the CBF conditions (with a user-specified robustness margin). We evaluate the risk incurred by the trade-off between the robustness margin of the system and its performance using the formalism of risk measures. We demonstrate our approach on challenging nonlinear control examples such as quadcopter motion planning and a unicycle.

Risk-Awareness in Learning Neural Controllers for Temporal Logic Objectives

Conformance is defined as a measure of distance between the behaviors of two dynamical systems. The notion of conformance can accelerate system design when models of varying fidelities are available on which analysis and control design can be done more efficiently. Ultimately, conformance can capture distance between design models and their real implementations and thus aid in robust system design. In this paper, we are interested in the conformance of stochastic dynamical systems. We argue that probabilistic reasoning over the distribution of distances between model trajectories is a good measure for stochastic conformance. Additionally, we propose the non-conformance risk to reason about the risk of stochastic systems not being conformant. We show that both notions have the desirable transference property, meaning that conformant systems satisfy similar system specifications, i.e., if the first model satisfies a desirable specification, the second model will satisfy (nearly) the same specification. Lastly, we propose how stochastic conformance and the non-conformance risk can be estimated from data using statistical tools such as conformal prediction. We present empirical evaluations of our method on an F-16 aircraft, an autonomous vehicle, a spacecraft, and Dubin’s vehicle.

pdf/conformance-testing-for-stochastic-cyber-physical-systems-3k6cv94dx9.pdf

Conformance Testing for Stochastic Cyber-Physical Systems

Lane keep assist (LKA) and Adaptive Cruise Control (ACC) are two fundamental yet critical functions for autonomous driving, and conventional methods using PID controllers may not perform well in certain extreme driving conditions. In this paper, we propose a reinforcement learning based approach to train the agent to learn LKA and ACC and hence adapt to diverse scenarios. Particularly, we employ deep deterministic policy gradient (DDPG) algorithm to train the agent and consider both state space and action space as continuous, and designed two neural network critic-network and actor-network to simulate the strategy function and Q-function. Then, we train the two neural networks by deep learning method. Finally, Simulations are conducted with both reinforcement learning and traditional PID controller, and the results of reinforcement learning is more adaptive to extreme road conditions in comparison with a traditional PID controller.

Path tracking control based on Deep reinforcement learning in Autonomous driving

The wide availability of data coupled with the computational advances in artificial intelligence and machine learning promise to enable many future technologies such as autonomous driving. While there has been a variety of successful demonstrations of these technologies, critical system failures have repeatedly been reported. Even if rare, such system failures pose a serious barrier to adoption without a rigorous risk assessment. This article presents a framework for the systematic and rigorous risk verification of systems. We consider a wide range of system specifications formulated in signal temporal logic (STL) and model the system as a stochastic process, permitting discrete-time and continuous-time stochastic processes. We then define the STL robustness risk as the risk of lacking robustness against failure. This definition is motivated as system failures are often caused by missing robustness to modeling errors, system disturbances, and distribution shifts in the underlying data generating process. Within the definition, we permit general classes of risk measures and focus on tail risk measures such as the value-at-risk and the conditional value-at-risk. While the STL robustness risk is in general hard to compute, we propose the approximate STL robustness risk as a more tractable notion that upper bounds the STL robustness risk. We show how the approximate STL robustness risk can accurately be estimated from system trajectory data. For discrete-time stochastic processes, we show under which conditions the approximate STL robustness risk can even be computed exactly. We illustrate our verification algorithm in the autonomous driving simulator CARLA and show how a least risky controller can be selected among four neural network lane-keeping controllers for five meaningful system specifications.

https://export.arxiv.org/pdf/2205.14523v2.pdf

Risk of Stochastic Systems for Temporal Logic Specifications

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Le Jiang

Author Tools

Chat about Author

Papers

Path tracking control based on Deep reinforcement learning in Autonomous driving

Risk of Stochastic Systems for Temporal Logic Specifications