This paper presents a comprehensive literature review on applications of deep reinforcement learning (DRL) in communications and networking. Modern networks, e.g., Internet of Things (IoT) and unmanned aerial vehicle (UAV) networks, become more decentralized and autonomous. In such networks, network entities need to make decisions locally to maximize the network performance under uncertainty of network environment. Reinforcement learning has been efficiently used to enable the network entities to obtain the optimal policy including, e.g., decisions or actions, given their states when the state and action spaces are small. However, in complex and large-scale networks, the state and action spaces are usually large, and the reinforcement learning may not be able to find the optimal policy in reasonable time. Therefore, DRL, a combination of reinforcement learning with deep learning, has been developed to overcome the shortcomings. In this survey, we first give a tutorial of DRL from fundamental concepts to advanced models. Then, we review DRL approaches proposed to address emerging issues in communications and networking. The issues include dynamic network access, data rate control, wireless caching, data offloading, network security, and connectivity preservation which are all important to next generation networks, such as 5G and beyond. Furthermore, we present applications of DRL for traffic routing, resource sharing, and data collection. Finally, we highlight important challenges, open issues, and future research directions of applying DRL.

Applications of Deep Reinforcement Learning in Communications and Networking: A Survey

The rapid uptake of mobile devices and the rising popularity of mobile applications and services pose unprecedented demands on mobile and wireless networking infrastructure. Upcoming 5G systems are evolving to support exploding mobile traffic volumes, real-time extraction of fine-grained analytics, and agile management of network resources, so as to maximize user experience. Fulfilling these tasks is challenging, as mobile environments are increasingly complex, heterogeneous, and evolving. One potential solution is to resort to advanced machine learning techniques, in order to help manage the rise in data volumes and algorithm-driven applications. The recent success of deep learning underpins new and powerful tools that tackle problems in this space. In this paper, we bridge the gap between deep learning and mobile and wireless networking research, by presenting a comprehensive survey of the crossovers between the two areas. We first briefly introduce essential background and state-of-the-art in deep learning techniques with potential applications to networking. We then discuss several techniques and platforms that facilitate the efficient deployment of deep learning onto mobile systems. Subsequently, we provide an encyclopedic review of mobile and wireless networking research based on deep learning, which we categorize by different domains. Drawing from our experience, we discuss how to tailor deep learning to mobile environments. We complete this survey by pinpointing current challenges and open future directions for research.

/pdf/deep-learning-in-mobile-and-wireless-networking-a-survey-ct8cy2kuqi.pdf

Deep Learning in Mobile and Wireless Networking: A Survey

Massive MIMO is a reality—What is next?: Five promising research directions for antenna arrays

This paper deals with the use of emerging deep learning techniques in future wireless communication networks. It will be shown that the data-driven approaches should not replace, but rather complement, traditional design techniques based on mathematical models. Extensive motivation is given for why deep learning based on artificial neural networks will be an indispensable tool for the design and operation of future wireless communication networks, and our vision of how artificial neural networks should be integrated into the architecture of future wireless communication networks is presented. A thorough description of deep learning methodologies is provided, starting with the general machine learning paradigm, followed by a more in-depth discussion about deep learning and artificial neural networks, covering the most widely used artificial neural network architectures and their training methods. Deep learning will also be connected to other major learning frameworks, such as reinforcement learning and transfer learning. A thorough survey of the literature on deep learning for wireless communication networks is provided, followed by a detailed description of several novel case studies wherein the use of deep learning proves extremely useful for network design. For each case study, it will be shown how the use of (even approximate) mathematical models can significantly reduce the amount of live data that needs to be acquired/measured to implement the data-driven approaches. Finally, concluding remarks describe those that, in our opinion, are the major directions for future research in this field.

pdf/wireless-networks-design-in-the-era-of-deep-learning-model-4gu4bgrl7j.pdf

Wireless Networks Design in the Era of Deep Learning: Model-Based, AI-Based, or Both?

The upcoming fifth generation (5G) of wireless networks is expected to lay a foundation of intelligent networks with the provision of some isolated artificial intelligence (AI) operations. However, fully intelligent network orchestration and management for providing innovative services will only be realized in Beyond 5G (B5G) networks. To this end, we envisage that the sixth generation (6G) of wireless networks will be driven by on-demand self-reconfiguration to ensure a many-fold increase in the network performance and service types. The increasingly stringent performance requirements of emerging networks may finally trigger the deployment of some interesting new technologies, such as large intelligent surfaces, electromagnetic–orbital angular momentum, visible light communications, and cell-free communications, to name a few. Our vision for 6G is a massively connected complex network capable of rapidly responding to the users’ service calls through real-time learning of the network state as described by the network edge (e.g., base-station locations and cache contents), air interface (e.g., radio spectrum and propagation channel), and the user-side (e.g., battery-life and locations). The multi-state, multi-dimensional nature of the network state, requiring the real-time knowledge, can be viewed as a quantum uncertainty problem. In this regard, the emerging paradigms of machine learning (ML), quantum computing (QC), and quantum ML (QML) and their synergies with communication networks can be considered as core 6G enablers. Considering these potentials, starting with the 5G target services and enabling technologies, we provide a comprehensive review of the related state of the art in the domains of ML (including deep learning), QC, and QML and identify their potential benefits, issues, and use cases for their applications in the B5G networks. Subsequently, we propose a novel QC-assisted and QML-based framework for 6G communication networks while articulating its challenges and potential enabling technologies at the network infrastructure, network edge, air interface, and user end. Finally, some promising future research directions for the quantum- and QML-assisted B5G networks are identified and discussed.

/pdf/quantum-machine-learning-for-6g-communication-networks-state-1zr3shv6z6.pdf

Quantum Machine Learning for 6G Communication Networks: State-of-the-Art and Vision for the Future

Algorithms for Massive MIMO uplink detection and downlink precoding typically rely on a centralized approach, by which baseband data from all antenna modules are routed to a central node in order to be processed. In the case of Massive MIMO, where hundreds or thousands of antennas are expected in the base-station, said routing becomes a bottleneck since interconnection throughput is limited. This paper presents a fully decentralized architecture and an algorithm for Massive MIMO uplink detection and downlink precoding based on the Coordinate Descent (CD) method, which does not require a central node for these tasks. Through a recursive approach and very low complexity operations, the proposed algorithm provides a good trade-off between performance, interconnection throughput and latency. Further, our proposed solution achieves significantly lower interconnection data-rate than other architectures, enabling future scalability.

Decentralized Massive MIMO Processing Exploring Daisy-Chain Architecture and Recursive Algorithms

Massive MIMO (MaMI) is often promoted as a technology that will enable the use of low-quality, cheap hardware. One particular component that has been in the focus of MaMI-related research is the analog-to-digital converter (ADC), and use of very low-resolution ADCs has been proposed. However, studies about whether this strategy is justified from an energy-efficiency point of view have largely been inconclusive. In this paper, we choose system setup and models that reflect the hardware implementation reality as close as possible and perform a parametric analysis of uplink energy efficiency as a function of ADC resolution. If antenna scaling and decrease of ADC resolution are considered independently, the energy efficiency is shown to be maximized at intermediate ADC resolutions, typically in the range of 4–8 bits. Moreover, optimal ADC resolution does not decrease when more antennas are used except in some specific cases, and when it does, the decrease is approximately logarithmic in the number of antennas. In the case when antenna scaling and ADC degradation are coupled through a constant-performance constraint, it is shown that energy efficiency cannot improve with reduced bit resolution unless the power consumption of blocks other than ADCs scales down with the upscaling of antennas at a fast enough rate. Altogether it is concluded that in MaMI, intermediate ADC resolutions are optimal in energy efficiency sense, and, except in some special cases, scaling up the antennas to very large numbers does not change this conclusion.

When Are Low Resolution ADCs Energy Efficient in Massive MIMO

6G will be characterized by extreme use cases, not only for communication, but also for localization, and sensing. The use cases can be directly mapped to requirements in terms of standard key performance indicators (KPIs), such as data rate, latency, or localization accuracy. The goal of this paper is to go one step further and map these standard KPIs to requirements on signals, on hardware architectures, and on deployments. Based on this, system solutions can be identified that can support several use cases simultaneously. Since there are several ways to meet the KPIs, there is no unique solution and preferable configurations will be discussed.

https://zenodo.org/record/7078754/files/Hexa_X_WP2_and_WP3_EUCNC2022.pdf

6G Radio Requirements to Support Integrated Communication, Localization, and Sensing

Algorithms for Massive MIMO uplink detection typically rely on a centralized approach, by which baseband data from all antennas modules are routed to a central node in order to be processed. In case of Massive MIMO, where hundreds or thousands of antennas are expected in the base-station, this architecture leads to a bottleneck, with critical limitations in terms of interconnection bandwidth requirements. This paper presents a fully decentralized architecture and algorithms for Massive MIMO uplink based on recursive methods, which do not require a central node for the detection process. Through a recursive approach and very low complexity operations, the proposed algorithms provide a sequence of estimates that converge asymptotically to the zero-forcing solution, without the need of specific hardware for matrix inversion. The proposed solution achieves significantly lower interconnection data-rate than other architectures, enabling future scalability.

Fully Decentralized Massive MIMO Detection Based on Recursive Methods

This paper provides an initial investigation on the application of convolutional neural networks (CNNs) for fingerprint-based positioning using measured massive MIMO channels. When represented in appropriate domains, massive MIMO channels have a sparse structure which can be efficiently learned by CNNs for positioning purposes. We evaluate the positioning accuracy of state-of-the-art CNNs with channel fingerprints generated from a channel model with a rich clustered structure: the COST 2100 channel model. We find that moderately deep CNNs can achieve fractional-wavelength positioning accuracies, provided that an enough representative data set is available for training.

Deep Convolutional Neural Networks for Massive MIMO Fingerprint-Based Positioning

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Muris Sarajlic

Author Tools

Chat about Author