Model-free deep reinforcement learning (RL) algorithms have been demonstrated on a range of challenging decision making and control tasks. However, these methods typically suffer from two major challenges: very high sample complexity and brittle convergence properties, which necessitate meticulous hyperparameter tuning. Both of these challenges severely limit the applicability of such methods to complex, real-world domains. In this paper, we propose soft actor-critic, an off-policy actor-critic deep RL algorithm based on the maximum entropy reinforcement learning framework. In this framework, the actor aims to maximize expected reward while also maximizing entropy. That is, to succeed at the task while acting as randomly as possible. Prior deep RL methods based on this framework have been formulated as Q-learning methods. By combining off-policy updates with a stable stochastic actor-critic formulation, our method achieves state-of-the-art performance on a range of continuous control benchmark tasks, outperforming prior on-policy and off-policy methods. Furthermore, we demonstrate that, in contrast to other off-policy algorithms, our approach is very stable, achieving very similar performance across different random seeds.

Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor

A review of shape memory alloy research, applications and opportunities

Simultaneous localization and mapping (SLAM) consists in the concurrent construction of a model of the environment (the map ), and the estimation of the state of the robot moving within it. The SLAM community has made astonishing progress over the last 30 years, enabling large-scale real-world applications and witnessing a steady transition of this technology to industry. We survey the current state of SLAM and consider future directions. We start by presenting what is now the de-facto standard formulation for SLAM. We then review related work, covering a broad set of topics including robustness and scalability in long-term mapping, metric and semantic representations for mapping, theoretical performance guarantees, active SLAM and exploration, and other new frontiers. This paper simultaneously serves as a position paper and tutorial to those who are users of SLAM. By looking at the published research with a critical eye, we delineate open challenges and new research issues, that still deserve careful scientific investigation. The paper also contains the authors’ take on two questions that often animate discussions during robotics conferences: Do robots need SLAM? and Is SLAM solved?

/pdf/past-present-and-future-of-simultaneous-localization-and-27i1pn9khc.pdf

Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age

Model-free deep reinforcement learning (RL) algorithms have been successfully applied to a range of challenging sequential decision making and control tasks. However, these methods typically suffer from two major challenges: high sample complexity and brittleness to hyperparameters. Both of these challenges limit the applicability of such methods to real-world domains. In this paper, we describe Soft Actor-Critic (SAC), our recently introduced off-policy actor-critic algorithm based on the maximum entropy RL framework. In this framework, the actor aims to simultaneously maximize expected return and entropy. That is, to succeed at the task while acting as randomly as possible. We extend SAC to incorporate a number of modifications that accelerate training and improve stability with respect to the hyperparameters, including a constrained formulation that automatically tunes the temperature hyperparameter. We systematically evaluate SAC on a range of benchmark tasks, as well as real-world challenging tasks such as locomotion for a quadrupedal robot and robotic manipulation with a dexterous hand. With these improvements, SAC achieves state-of-the-art performance, outperforming prior on-policy and off-policy methods in sample-efficiency and asymptotic performance. Furthermore, we demonstrate that, in contrast to other off-policy algorithms, our approach is very stable, achieving similar performance across different random seeds. These results suggest that SAC is a promising candidate for learning in real-world robotics tasks.

pdf/soft-actor-critic-algorithms-and-applications-rskq0fyv6a.pdf

Soft Actor-Critic Algorithms and Applications

Combining visual and inertial measurements has become popular in mobile robotics, since the two sensing modalities offer complementary characteristics that make them the ideal choice for accurate visual-inertial odometry or simultaneous localization and mapping SLAM. While historically the problem has been addressed with filtering, advancements in visual estimation suggest that nonlinear optimization offers superior accuracy, while still tractable in complexity thanks to the sparsity of the underlying problem. Taking inspiration from these findings, we formulate a rigorously probabilistic cost function that combines reprojection errors of landmarks and inertial terms. The problem is kept tractable and thus ensuring real-time operation by limiting the optimization to a bounded window of keyframes through marginalization. Keyframes may be spaced in time by arbitrary intervals, while still related by linearized inertial terms. We present evaluation results on complementary datasets recorded with our custom-built stereo visual-inertial hardware that accurately synchronizes accelerometer and gyroscope measurements with imagery. A comparison of both a stereo and monocular version of our algorithm with and without online extrinsics estimation is shown with respect to ground truth. Furthermore, we compare the performance to an implementation of a state-of-the-art stochastic cloning sliding-window filter. This competitive reference implementation performs tightly coupled filtering-based visual-inertial odometry. While our approach declaredly demands more computation, we show its superior performance in terms of accuracy.

/pdf/keyframe-based-visual-inertial-odometry-using-nonlinear-17tj6999kv.pdf

Keyframe-based visual-inertial odometry using nonlinear optimization

Presented at Robotics: Science and Systems VIII, July 09-July 13, 2012, University of Sydney, Sydney, NSW, Australia.

http://www.roboticsproceedings.org/rss08/p44.pdf

Walking and Running on Yielding and Fluidizing Ground

This work presents the design, fabrication, and testing of a novel hexapedal walking millirobot using only two actuators. Fabricated from S2-glass reinforced composites and flexible polymer hinges using the smart composite microstructures (SCM) process, the robot is capable of speeds up to 1 body length/sec or approximately 3 cm/s. All power and control electronics are onboard and remote commands are enabled by an IrDA link. Actuation is provided by shape memory alloy wire. At 2.4 g including control electronics and battery, RoACH is the smallest and lightest autonomous legged robot produced to date.

RoACH: An autonomous 2.4g crawling hexapod robot

Rapidly running arthropods like cockroaches make use of passive dynamics to achieve remarkable locomotion performance with regard to stability, speed, and maneuverability In this work, we take inspiration from these organisms to design, fabricate, and control a 10cm, 24 gram underactuated hexapedal robot capable of running at 14 body lengths per second and performing dynamic turning maneuvers Our design relies on parallel kinematic mechanisms fabricated using the scaled smart composite microstructures (SCM) process and viscoelastic polymer legs with tunable stiffness In addition to the novel robot design, we present experimental validation of the lateral leg spring (LLS) locomotion model's prediction that dynamic turning can be achieved by modulating leg stiffness Finally, we present and validate a leg design for active stiffness control using shape memory alloy and demonstrate the ability of the robot to execute near-gymnastic 90° turns in the span of five strides

Bio-inspired design and dynamic maneuverability of a minimally actuated six-legged robot

We present a set of tools and a process, making use of inexpensive and environmentally friendly materials, that enable the rapid realization of fully functional large scale prototypes of folded mobile millirobots. By mimicking the smart composite microstructure (SCM) process at a 2-10X scale using posterboard, and commonly available polymer films, we can realize a prototype design in a matter of minutes compared with days for a complicated SCM design at the small scale. The time savings enable a significantly shorter design cycle by allowing for immediate discovery of design flaws and introduction of design improvements prior to beginning construction at the small scale. In addition, the technology eases the difficulty of visualizing and creating folded 3D structures from 2D parts. We use the example of a fully functional hexapedal crawling robot design to illustrate the process and to verify a scaling law which we propose.

/pdf/fast-scale-prototyping-for-folded-millirobots-1vpfbkb1pe.pdf

Fast scale prototyping for folded millirobots

We study the locomotor mechanics of a small, lightweight robot (DynaRoACH, 10 cm, 25 g) which can move on a granular substrate of 3 mm diameter glass particles at speeds up to 5 body length/s, approaching the performance of certain desert-dwelling animals. To reveal how the robot achieves this performance, we used high-speed imaging to capture its kinematics, and developed a numerical multi-body simulation of the robot coupled to an experimentally validated simulation of the granular medium. Average speeds measured in experiment and simulation agreed well, and increased nonlinearly with stride frequency, reflecting a change in propulsion mode. At low frequencies, the robot used a quasi-static rotary walking mode, in which the substrate yielded as legs penetrated and then solidified once vertical force balance was achieved. At high frequencies the robot propelled itself using the speed-dependent fluid-like inertial response of the material. The simulation allows variation of parameters which are inconvenient to modify in experiment, and thus gives insight into how substrate and robot properties change performance. Our study reveals how lightweight animals can achieve high performance on granular substrates; such insights can advance the design and control of robots in deformable terrains.

/pdf/ground-fluidization-promotes-rapid-running-of-a-lightweight-f2u36bfvhe.pdf

Ground fluidization promotes rapid running of a lightweight robot

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Aaron M. Hoover

Author Tools

Chat about Author