In recent years, the advancement of artificial intelligence techniques has led to significant interest in reinforcement learning (RL) within the traffic and transportation community. Dynamic traffic control has emerged as a prominent application field for RL in traffic systems. This paper presents a comprehensive survey of RL studies in dynamic traffic control, addressing the challenges associated with implementing RL-based traffic control strategies in practice, and identifying promising directions for future research. The first part of this paper provides a comprehensive overview of existing studies on RL-based traffic control strategies, encompassing their model designs, training algorithms, and evaluation methods. It is found that only a few studies have isolated the training and testing environments while evaluating their RL controllers. Subsequently, we examine the challenges involved in implementing existing RL-based traffic control strategies. We investigate the learning costs associated with online RL methods and the transferability of offline RL methods through simulation experiments. The simulation results reveal that online training methods with random exploration suffer from high exploration and learning costs. Additionally, the performance of offline RL methods is highly reliant on the accuracy of the training simulator. These limitations hinder the practical implementation of existing RL-based traffic control strategies. The final part of this paper summarizes and discusses a few existing efforts which attempt to overcome these challenges. This review highlights a rising volume of studies dedicated to mitigating the limitations of RL strategies, with the specific aim of enhancing their practical implementation in recent years. 

Leveraging reinforcement learning for dynamic traffic control: A survey and challenges for field implementation

Managing traffic flow at intersections in a large-scale network remains challenging. Multi-modal signalized intersections integrate various objectives, including minimizing the queue length and maintaining constant bus headway. Inefficient traffic signals and bus headway control strategies may cause severe traffic jams, high delays for bus passengers, and bus bunching that harms bus line operations. To simultaneously improve the level of service for car traffic and the bus system in a multi-modal network, this paper integrates bus priority and holding with traffic signal control via decentralized controllers based on Reinforcement Learning (RL). The controller agents act and learn from a synthetic traffic environment built with the microscopic traffic simulator SUMO. Action information is shared among agents to achieve cooperation, forming a Multi-Agent Reinforcement Learning (MARL) framework. The agents simultaneously aim to minimize vehicles' total stopping time and homogenize the forward and backward space headways for buses approaching intersections at each decision step. The Deep Q-Network (DQN) algorithm is applied to manage the continuity of the state space. The tradeoff between the bus transit and car traffic objectives is discussed using various numerical experiments. The introduced method is tested in scenarios with distinct bus lane layouts and bus line deployments. The proposed controller outperforms model-based adaptive control methods and the centralized RL method regarding global traffic efficiency and bus transit stability. Furthermore, the remarkable scalability and transferability of trained models are demonstrated by applying them to several different test networks without retraining. 

Decentralized signal control for multi-modal traffic network: A deep reinforcement learning approach

Bottleneck areas are prone to severe traffic congestion due to the sudden drop in capacity. To improve traffic efficiency in the bottleneck area, this paper proposes a multi-agent deep reinforcement learning framework integrating collision avoidance strategies to improve traffic efficiency in a mandatory lane change scenario. The proposed method considers distance-keeping and lane-changing coordination in a connected autonomous vehicle (CAV) environment, by controlling vehicles' longitudinal and lateral movement to effectively reduce traffic congestion in a mandatory lane change scenario. This framework was trained and tested in a simulation environment that is the same as the natural driving environment. Compared with real-world data and the benchmark model (a Dueling Double Deep Q-Network-based model), the proposed model shows better performance in terms of average speed, travel time, throughput, and safety in the bottleneck area. The results show that the proposed model can effectively reduce traffic congestion and improve traffic efficiency in a mandatory lane change scenario. 

A multi-agent reinforcement learning-based longitudinal and lateral control of CAVs to improve traffic efficiency in a mandatory lane change scenario

Most of the current variable speed limit (VSL) strategies are designed to alleviate congestion in relatively short freeway segments with a single bottleneck. However, in reality, consecutive bottlenecks can occur simultaneously due to the merging flow from multiple ramps. In such situations, the existing strategies use multiple VSL controllers that operate independently, without considering the traffic flow interactions and speed limit differences. In this research, we introduced a multiagent reinforcement learning-based VSL (MARL-VSL) approach to enhance collaboration among VSL controllers. The MARL-VSL approach employed a centralized training with decentralized execution structure to achieve a joint optimal solution for a series of VSL controllers. The consecutive bottleneck scenarios were simulated in the modified cell transmission model to validate the effectiveness of the proposed strategy. An independent single-agent reinforcement learning-based VSL (ISARL-VSL) and a feedback-based VSL (feedback-VSL) were also applied for comparison. Time-varying heterogeneous traffic flow stemming from the mainline and ramps was loaded into the freeway network. The results demonstrated that the proposed MARL-VSL achieved superior performance compared to the baseline methods. The proposed approach reduced the total time spent by the vehicles by 18.01% and 17.07% in static and dynamic traffic scenarios, respectively. The control actions of the MARL-VSL were more appropriate in maintaining a smooth freeway traffic flow due to its superior collaboration performance. More specifically, the MARL-VSL significantly improved the average driving speed and speed homogeneity across the entire freeway.

/pdf/coordinated-variable-speed-limit-control-for-consecutive-3vkvggbb.pdf

Coordinated Variable Speed Limit Control for Consecutive Bottlenecks on Freeways Using Multiagent Reinforcement Learning

Data trading has been hindered by privacy concerns associated with user-owned data and the infinite reproducibility of data, making it challenging for data owners to retain exclusive rights over their data once it has been disclosed. Traditional data pricing models relied on uniform pricing or subscription-based models. However, with the development of Privacy-Preserving Computing techniques, the market can now protect the privacy and complete transactions using progressively disclosed information, which creates a technical foundation for generating greater social welfare through data usage. In this study, we propose a novel approach to modeling multi-round data trading with progressively disclosed information using a matchmaking-based Markov Decision Process (MDP) and introduce a Social Welfare-optimized Data Pricing Mechanism (SWDPM) to find optimal pricing strategies. To the best of our knowledge, this is the first study to model multi-round data trading with progressively disclosed information. Numerical experiments demonstrate that the SWDPM can increase social welfare 3 times by up to 54 % in trading feasibility, 43 % in trading efficiency, and 25 % in trading fairness by encouraging better matching of demand and price negotiation among traders. 

SWDPM: A Social Welfare-Optimized Data Pricing Mechanism

Conventional reinforcement learning (RL) models of variable speed limit (VSL) control systems (and traffic control systems in general) cannot be trained in real traffic process because new control actions are usually explored randomly, which may result in high costs (delays) due to exploration and learning. For this reason, existing RL-based VSL control approaches need a traffic simulator for training. However, the performance of those approaches are dependent on the accuracy of the simulators. This paper proposes a new RL-based VSL control approach to overcome the aforementioned problems. The proposed VSL control approach is designed to improve traffic efficiency by using VSLs against freeway jam waves. It applies an iterative training framework, where the optimal control policy is updated by exploring new control actions both online and offline in each iteration. The explored control actions are evaluated in real traffic process, thus it avoids that the RL model learns only from a traffic simulator. The proposed VSL control approach is tested using a macroscopic traffic simulation model to represent real world traffic flow dynamics. By comparing with existing VSL control approaches, the proposed approach is demonstrated to have advantages in the following two aspects: (i) it alleviates the impact of model mismatch, which occurs in both model-based VSL control approaches and existing RL-based VSL control approaches, via replacing knowledge from the models by knowledge from the real process, and (ii) it significantly reduces the exploration and learning costs compared to existing RL-based VSL control approaches. • An RL-based VSL control approach against freeway jam waves is developed. • The proposed method alleviates model mismatch via online/offline learning. • The proposed method has less exploration and learning costs than random exploration. • The proposed method is compared with various of existing VSL approaches. 

A new reinforcement learning-based variable speed limit control approach to improve traffic efficiency against freeway jam waves

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Le Zhang

Author Tools

Chat about Author

Papers

A new reinforcement learning-based variable speed limit control approach to improve traffic efficiency against freeway jam waves