Journal Article10.1109/jsac.2022.3228558
Deep Reinforcement Learning Based Computation Offloading and Trajectory Planning for Multi-UAV Cooperative Target Search
51
TL;DR: In this paper , a deep reinforcement learning (DRL) technique was proposed to jointly make optimal computation offloading decisions and flying orientation choices for multi-UAV cooperative target search, and extensive simulations validate the effectiveness of the proposed techniques, and comprehensive discussions on how different parameters affect the search performance are given.
read more
Abstract: Unmanned aerial vehicles (UAVs) are widely used for surveillance and monitoring to complete target search tasks. However, the short battery life and moderate computational capability hinder UAVs to process computation-intensive tasks. The emerging edge computing technologies can alleviate this problem by offloading tasks to the ground edge servers. How to evaluate the search process so as to make optimal offloading decisions and make optimal flying trajectories represent fundamental research challenges. In this paper, we propose to utilize the concept of uncertainty to evaluate the search process, which reflects the reliability of the target search results. Thereafter, we propose a deep reinforcement learning (DRL) technique to jointly make optimal computation offloading decisions and flying orientation choices for multi-UAV cooperative target search. Specifically, we first formulate an uncertainty minimization problem based on the established system model. By introducing a reward function, we prove that the uncertainty minimization problem is equivalent to a reward maximization problem, which is further analyzed by a Markov decision process (MDP). To obtain the optimal task offloading decisions and flying orientation choices, a deep Q-network (DQN) based DRL architecture with a separated Q-network is then proposed. Finally, extensive simulations validate the effectiveness of the proposed techniques, and comprehensive discussions on how different parameters affect the search performance are given.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
Mobile Edge Computing and Machine Learning in The Internet of Unmanned Aerial Vehicles: A Survey
TL;DR: In this paper , the authors provide a comprehensive review of key technologies, applications, solutions and challenges based on the integration of Mobile Edge Computing (MEC) and Machine Learning (ML) in the Internet of UAVs.
53
Resource Allocation and Offloading Strategy for UAV-Assisted LEO Satellite Edge Computing
TL;DR: In this paper, the authors investigated the computational tasks and resource allocation in a UAV-assisted multi-layer LEO satellite network, taking into account satellite computing resources and device task volumes.
41
When Moving Target Defense Meets Attack Prediction in Digital Twins: A Convolutional and Hierarchical Reinforcement Learning Approach
Tao Zhang,Changqiao Xu,Yibo Lian,Haijiang Tian,Jiawen Kang,Xiao-feng Kuang,Dusit Niyato +6 more
TL;DR: A collaborative mutation-based MTD (CM-MTD) in DTMN is proposed, which mainly considers two MTD schemes called host address mutation (HAM) and route mutation (RM), respectively, which adjust network properties and invalidate different stages of cyber kill chain.
28
UAV Swarm Cooperative Target Search: A Multi-Agent Reinforcement Learning Approach
TL;DR: This paper proposes a distributed collaborative search method based on a multi-agent reinforcement learning algorithm, which can operate efficiently in complex and large-scale scenarios and can utilize a convolutional neural network to process high-dimensional map data with almost no loss of the structure information.
19
Enhancing Multi-UAV Reconnaissance and Search Through Double Critic DDPG With Belief Probability Maps
Boquan Zhang,Xiang Lin,Yifan Zhu,Jing Tian,Zhi Zhu +4 more
TL;DR: This paper proposes Double Critic DDPG (DCDDPG) for multi-UAV reconnaissance and search, utilizing belief probability maps and Decentralized POMDP to optimize coverage and target localization, outperforming existing techniques in search efficiency and coverage.
16
References
Human-level control through deep reinforcement learning
Volodymyr Mnih,Koray Kavukcuoglu,David Silver,Andrei Rusu,Joel Veness,Marc G. Bellemare,Alex Graves,Martin Riedmiller,Andreas K. Fidjeland,Georg Ostrovski,Stig Petersen,Charles Beattie,Amir Sadik,Ioannis Antonoglou,Helen King,Dharshan Kumaran,Daan Wierstra,Shane Legg,Demis Hassabis +18 more
TL;DR: This work bridges the divide between high-dimensional sensory inputs and actions, resulting in the first artificial agent that is capable of learning to excel at a diverse array of challenging tasks.
Deep reinforcement learning with double Q-learning
H Van Hasselt,Arthur Guez,David Silver +2 more
- 01 Jan 2015
TL;DR: In this article, the authors show that the DQN algorithm suffers from substantial overestimation in some games in the Atari 2600 domain, and they propose a specific adaptation to the algorithm and show that this algorithm not only reduces the observed overestimations, but also leads to much better performance on several games.
7.9K
•Proceedings Article
Continuous control with deep reinforcement learning
Timothy P. Lillicrap,Jonathan J. Hunt,Alexander Pritzel,Nicolas Heess,Tom Erez,Yuval Tassa,David Silver,Daan Wierstra +7 more
- 22 Jul 2016
TL;DR: In this paper, an actor-critic, model-free algorithm based on the deterministic policy gradient is proposed to operate over continuous action spaces, which is able to find policies whose performance is competitive with those found by a planning algorithm with full access to the dynamics of the domain.
6.5K
Efficient Multi-User Computation Offloading for Mobile-Edge Cloud Computing
TL;DR: In this article, a game theoretic approach for computation offloading in a distributed manner was adopted to solve the multi-user offloading problem in a multi-channel wireless interference environment.
2.8K
Deep Learning With Edge Computing: A Review
Jiasi Chen,Xukan Ran +1 more
- 15 Jul 2019
TL;DR: This paper will provide an overview of applications where deep learning is used at the network edge, discuss various approaches for quickly executing deep learning inference across a combination of end devices, edge servers, and the cloud, and describe the methods for training deep learning models across multiple edge devices.
1.3K