In recent years, significant progress has been made in solving challenging problems across various domains using deep reinforcement learning (RL). Reproducing existing work and accurately judging the improvements offered by novel methods is vital to sustaining this progress. Unfortunately, reproducing results for state-of-the-art deep RL methods is seldom straightforward. In particular, non-determinism in standard benchmark environments, combined with variance intrinsic to the methods, can make reported results tough to interpret. Without significance metrics and tighter standardization of experimental reporting, it is difficult to determine whether improvements over the prior state-of-the-art are meaningful. In this paper, we investigate challenges posed by reproducibility, proper experimental techniques, and reporting procedures. We illustrate the variability in reported metrics and results when comparing against common baselines and suggest guidelines to make future results in deep RL more reproducible. We aim to spur discussion about how to ensure continued progress in the field by minimizing wasted effort stemming from results that are non-reproducible and easily misinterpreted.

/pdf/deep-reinforcement-learning-that-matters-5b5xvmku1n.pdf

Deep Reinforcement Learning That Matters

Deep Reinforcement Learning that Matters

A Multi-Objective Deep Reinforcement Learning Framework

Motivated by the recent explosion of interest around Educational Robotics (ER), this paper attempts to re-approach this area by suggesting new ways of thinking and exploring the related concepts. The contribution of the paper is fourfold. First, future readers can use this paper as a reference point for exploring the expected learning outcomes of educational robotics. From an exhaustive list of potential learning gains, we propose a set of six learning outcomes that can offer a starting point for a viable model for the design of robotic activities. Second, the paper aims to serve as a survey for the most recent ER platforms. Driven by the growing number of available robotics platforms, we have gathered the most recent ER kits. We also propose a new way to categorize the platforms, free from their manufacturers’ vague age boundaries. The proposed categories, including No Code , Basic Code , and Advanced Code , are derived from the prior knowledge and the programming skills that a student needs to use them efficiently. Third, as the number of ER competitions, and tournaments increases in parallel with ER platforms’ increase, the paper presents and analyses the most popular robotic events. Robotics competitions encourage participants to develop and showcase their skills while promoting specific learning outcomes. The paper aims to provide an overview of those structures and discuss their efficacy. Finally, the paper explores the educational aspects of the presented ER competitions and their correlation with the six proposed learning outcomes. This raises the question of which primary features compose a competition and achieve its’ pedagogical goals. This paper is the first study that correlates potential learning gains with ER competitions to the best of our knowledge.

/pdf/educational-robotics-platforms-competitions-and-expected-14gjbr32p8.pdf

Educational Robotics: Platforms, Competitions and Expected Learning Outcomes

The development of a versatile, fully-capable humanoid robot as envisioned in science fiction books is one of the most challenging but interesting issues in the robotic field. Currently, existing humanoid robots are designed with different purposes and applications in mind. In humanoid robot development process, each robot is designed with various characteristics, abilities, and equipment, which influence the general structure, cost, and difficulty of development. Even though humanoid robot development is very popular, a few review papers are focusing on the design and development process of humanoid robots. Motivated by this, we present this review paper to show variations in the requirements, design, and development process and also propose a taxonomy of existing humanoid robots. It aims at demonstrating a general perspective of existing humanoid robots’ characteristics and applications. This paper includes state-of-the-art and successfully reported existing humanoid robot designs along with different robots used in various robot competitions.

A comprehensive survey on humanoid robot development

In 2019, Rhoban Football Club reached the first place of the KidSize soccer competition for the fourth time and performed the first in-game throw-in in the history of the Humanoid league. Building on our existing code-base, we improved some specific functionalities, introduced new behaviors and experimented with original methods for labeling videos. This paper presents and reviews our latest changes to both software and hardware, highlighting the lessons learned during RoboCup.

https://hal.archives-ouvertes.fr/hal-02330530/document

Rhoban Football Club: RoboCup Humanoid KidSize 2019 Champion Team Paper

Afin d'etre autonomes, les robots doivent etres capables de prendre des decisions en fonction des informations qu'ils percoivent de leur environnement. Cette these modelise les problemes de prise de decision robotique comme des processus de decision markoviens avec un espace d'etat et un espace d'action tous deux continus. Ce choix de modelisation permet de representer les incertitudes sur le resultat des actions appliquees par le robot. Les nouveaux algorithmes d'apprentissage presentes dans cette these se focalisent sur l'obtention de strategies applicables dans un domaine embarque. Ils sont appliques a deux problemes concrets issus de la RoboCup, une competition robotique internationale annuelle. Dans ces problemes, des robots humanoides doivent decider de la puissance et de la direction de tirs afin de maximiser les chances de marquer et controler la commande d'une primitive motrice pour preparer un tir.

Decision-making algorithms for autonomous robots

Rhoban Football Club: RoboCup Humanoid KidSize 2019 Champion Team Paper.

Odometry is an important element for the localization of mobile robots. For humanoid robots, it is very prone to integration errors, due to mechanical complexity, uncertainties and foot/ground contacts. Most of the time, a visual odometry is then used to encompass these problems. In this work we propose a method to compensate for odometry drifting using machine learning on a small size low-cost humanoid without vision. This method is tested on different ground conditions and exhibits a significant improvement in odometry accuracy.

Learning the odometry on a small humanoid robot

This paper presents a new method to learn online policies in continuous state, continuous action, model-free Markov decision processes, with two properties that are crucial for practical applications. First, the policies are implementable with a very low computational cost: once the policy is computed, the action corresponding to a given state is obtained in logarithmic time with respect to the number of samples used. Second, our method is versatile: it does not rely on any a priori knowledge of the structure of optimal policies. We build upon the Fitted Q-iteration algorithm which represents the $Q$-value as the average of several regression trees. Our algorithm, the Fitted Policy Forest algorithm (FPF), computes a regression forest representing the Q-value and transforms it into a single tree representing the policy, while keeping control on the size of the policy using resampling and leaf merging. We introduce an adaptation of Multi-Resolution Exploration (MRE) which is particularly suited to FPF. We assess the performance of FPF on three classical benchmarks for reinforcement learning: the "Inverted Pendulum", the "Double Integrator" and "Car on the Hill" and show that FPF equals or outperforms other algorithms, although these algorithms rely on the use of particular representations of the policies, especially chosen in order to fit each of the three problems. Finally, we exhibit that the combination of FPF and MRE allows to find nearly optimal solutions in problems where $\epsilon$-greedy approaches would fail.

Online Reinforcement Learning for Real-Time Exploration in Continuous State and Action Markov Decision Processes.

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Ludovic Hofer

Author Tools

Chat about Author