Evolutionary Multi-Objective Deep Reinforcement Learning for Autonomous UAV Navigation in Large-Scale Complex Environments

An，Guangyan; Wu，Ziyu; Shen，Zhilong; Shang，Ke; Ishibuchi，Hisao

10.1145/3583131.3590446

2023-07-15

Genetic and Evolutionary Computation Conference (GECCO)

GECCO 2023 - Proceedings of the 2023 Genetic and Evolutionary Computation Conference

633-641

JUL 15-19, 2023

null,Lisbon,PORTUGAL

1601 Broadway, 10th Floor, NEW YORK, NY, UNITED STATES

ASSOC COMPUTING MACHINERY

Autonomous navigation of Unmanned Aerial Vehicles (UAVs) in large-scale complex environments presents a significant challenge in modern aerospace engineering, as it requires effective decision-making in an environment with limited sensing capacity, dynamic changes, and dense obstacles. Reinforcement Learning (RL) has been applied in sequential control problems, but the manual setting of hyperparameters, including reward functions, often results in suboptimal solutions and inadequate training. To address these limitations, we propose a framework that combines Multi-Objective Evolutionary Algorithms (MOEAs) with RL algorithms. The proposed framework generates a set of non-dominating parameters for the reward function using MOEAs, leading to diverse decision-making preferences, efficient convergence, and improved performance. The framework was tested on the autonomous navigation of UAVs and demonstrated significant improvement compared to traditional RL methods. This work offers a novel perspective on the problem of autonomous UAV navigation in large-scale complex environments and highlights the potential for further improvement through the integration of RL and MOEAs.

First ; Corresponding

English

CPCI-S

National Natural Science Foundation of China["62002152","62250710163","62250710682"] ; Guangdong Provincial Key Laboratory[2020B121201001] ; Program for Guangdong Introducing Innovative and Enterpreneurial Teams[2017ZT07X386] ; Stable Support Plan Program of Shenzhen Natural Science Fund[20200925174447003] ; Shenzhen Science and Technology Program[KQTD2016 112514355531]

Computer Science

Computer Science, Artificial Intelligence ; Computer Science, Information Systems

WOS:001031455100071

2-s2.0-85167687529

Scopus

Guangdong Provincial Key Laboratory of Brain-inspired Intelligent Computation,Department of Computer Science and Engineering,Southern University of Science and Technology,Shenzhen,China

An，Guangyan,Wu，Ziyu,Shen，Zhilong,et al. Evolutionary Multi-Objective Deep Reinforcement Learning for Autonomous UAV Navigation in Large-Scale Complex Environments[C]. 1601 Broadway, 10th Floor, NEW YORK, NY, UNITED STATES:ASSOC COMPUTING MACHINERY,2023:633-641.