Multi-Agent Connected Autonomous Driving using Deep Reinforcement Learning
Palanisamy Praveen
- 19 Jul 2020
- pp 1-7
TL;DR: In this paper, the authors proposed the use of Partially Observable Markov Games (POSG) for formulating the connected autonomous driving problems with realistic assumptions, and provided a taxonomy of multi-agent learning environments based on the nature of tasks, nature of agents and the environment.
read more
Abstract: The capability to learn and adapt to changes in the driving environment is crucial for developing autonomous driving systems that are scalable beyond geo-fenced operational design domains. Deep Reinforcement Learning (RL) provides a promising and scalable framework for developing adaptive learning based solutions. Deep RL methods usually model the problem as a (Partially Observable) Markov Decision Process in which an agent acts in a stationary environment to learn an optimal behavior policy. However, driving involves complex interaction between multiple, intelligent (artificial or human) agents in a highly non-stationary environment. In this paper, we propose the use of Partially Observable Markov Games(POSG) for formulating the connected autonomous driving problems with realistic assumptions. We provide a taxonomy of multi-agent learning environments based on the nature of tasks, nature of agents and the nature of the environment to help in categorizing various autonomous driving problems that can be addressed under the proposed formulation. As our main contributions, we provide MACAD-Gym, a Multi-Agent Connected, Autonomous Driving agent learning platform for furthering research in this direction. Our MACAD-Gym platform provides an extensible set of Connected Autonomous Driving (CAD) simulation environments that enable the research and development of Deep RL- based integrated sensing, perception, planning and control algorithms for CAD systems with unlimited operational design domain under realistic, multi-agent settings. We also share the MACAD-Agents that were trained successfully using the MACAD-Gym platform to learn control policies for multiple vehicle agents in a partially observable, stop-sign controlled, 3-way urban intersection environment with raw (camera) sensor observations.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
Deep Reinforcement Learning for Autonomous Driving: A Survey
TL;DR: The authors provides a taxonomy of automated driving tasks where deep reinforcement learning (DRL) methods have been employed, while addressing key computational challenges in real world deployment of autonomous driving agents and delineates adjacent domains such as behavior cloning, imitation learning, inverse reinforcement learning that are related but are not classical RL algorithms.
•Posted Content
SMARTS: Scalable Multi-Agent Reinforcement Learning Training School for Autonomous Driving.
Ming Zhou,Jun Luo,Julian Villela,Yaodong Yang,David Rusu,Jiayu Miao,Weinan Zhang,Montgomery Alban,Iman Fadakar,Zheng Chen,Aurora Chongxi Huang,Ying Wen,Kimia Hassanzadeh,Daniel Graves,Dong Chen,Zhengbang Zhu,Nhat M. Nguyen,Mohamed A. Elsayed,Kun Shao,Sanjeevan Ahilan,Baokuan Zhang,Jiannan Wu,Zhengang Fu,Kasra Rezaee,Peyman Yadmellat,Mohsen Rohani,Nicolas Perez Nieves,Yihan Ni,Seyedershad Banijamali,Alexander Imani Cowen-Rivers,Zheng Tian,Daniel Palenicek,Haitham Bou-Ammar,Hongbo Zhang,Wulong Liu,Jianye Hao,Jun Wang +36 more
TL;DR: The design goals of SMARTS (Scalable Multi-Agent RL Training School) are described, its basic architecture and its key features are explained, and its use is illustrated through concrete multi-agent experiments on interactive scenarios.
Graph neural network and reinforcement learning for multi-agent cooperative control of connected autonomous vehicles
TL;DR: A connected autonomous vehicle (CAV) network can be defined as a set of connected vehicles including CAVs that operate on a specific spatial scope that may be a road network, corridor, or highway as discussed by the authors.
159
A review of reinforcement learning based energy management systems for electrified powertrains: Progress, challenge, and potential solution
Rujing Yin
- 01 Feb 2022
TL;DR: A comprehensive review of RL-based energy management strategies (EMS) is lacking in literature as discussed by the authors , however, a comprehensive survey of the literature can be found in our previous work, which reviewed the recent penetration of RL based EMS like Q-learning, Deep Q Learning, deep deterministic policy gradient in the electrified powertrains domain.
147
•Proceedings Article
Probabilistic recursive reasoning for multi-agent reinforcement learning
Ying Wen,Yaodong Yang,Rui Luo,Jun Wang,Wei Pan +4 more
- 09 May 2019
TL;DR: In this paper, a probabilistic recursive reasoning (PR2) framework for multi-agent reinforcement learning is introduced, where each agent can reason about how the opponents would react to its future behaviors.
References
•Posted Content
Playing Atari with Deep Reinforcement Learning
Volodymyr Mnih,Koray Kavukcuoglu,David Silver,Alex Graves,Ioannis Antonoglou,Daan Wierstra,Martin Riedmiller +6 more
TL;DR: This work presents the first deep learning model to successfully learn control policies directly from high-dimensional sensory input using reinforcement learning, which outperforms all previous approaches on six of the games and surpasses a human expert on three of them.
Markov games as a framework for multi-agent reinforcement learning
Michael L. Littman
- 10 Jul 1994
TL;DR: A Q-learning-like algorithm for finding optimal policies and its application to a simple two-player game in which the optimal policy is probabilistic is demonstrated.
Stochastic Games
TL;DR: In a stochastic game the play proceeds by steps from position to position, according to transition probabilities controlled jointly by the two players, and the expected total gain or loss is bounded by M, which depends on N 2 + N matrices.
3.1K
Dedicated Short-Range Communications (DSRC) Standards in the United States
John Kenney
- 16 Jun 2011
TL;DR: The content and status of the DSRC standards being developed for deployment in the United States are explained, including insights into why specific technical solutions are being adopted, and key challenges remaining for successful DSRC deployment.
2.1K
AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehicles
Shital Shah,Debadeepta Dey,Chris Lovett,Ashish Kapoor +3 more
- 15 May 2017
TL;DR: In this paper, the authors present a new simulator built on Unreal Engine that offers physically and visually realistic simulations for autonomous vehicles in real-world environments, including a physics engine that can operate at a high frequency for real-time hardware-in-the-loop (HITL) simulations with support for popular protocols (e.g., MavLink).