Enabling Surrogate-Assisted Evolutionary Reinforcement Learning via Policy Embedding
Evolutionary Reinforcement Learning (ERL) that applying Evolutionary Algorithms (EAs) to optimize the weight parameters of Deep Neural Network (DNN) based policies has been widely regarded as an alternative to traditional reinforcement learning methods. However, the evaluation of the iteratively generated population usually requires a large amount of computational time and can be prohibitively expensive, which may potentially restrict the applicability of ERL. Surrogate is often used to reduce the computational burden of evaluation in EAs. Unfortunately, in ERL, each individual of policy usually represents millions of weights parameters of DNN. This high-dimensional representation of policy has introduced a great challenge to the application of surrogates into ERL to speed up training. This paper proposes a PE-SAERL Framework to at the first time enable surrogate-assisted evolutionary reinforcement learning via policy embedding (PE). Empirical results on 5 Atari games show that the proposed method can perform more efficiently than the four state-of-the-art algorithms. The training process is accelerated up to 7x on tested games, comparing to its counterpart without the surrogate and PE.
First ; Corresponding
Cited Times [WOS]:0
|Document Type||Conference paper|
|Department||Department of Computer Science and Engineering|
1.Guangdong Provincial Key Laboratory of Brain-Inspired Intelligent Computation,Department of Computer Science and Engineering,Southern University of Science and Technology,Shenzhen,518055,China
2.Faculty of Engineering,Shenzhen MSU-BIT University,Shenzhen,518172,China
3.Research Institute of Trustworthy Autonomous Systems,Southern University of Science and Technology,Shenzhen,518055,China
4.Department of Statistics and Data Science,Southern University of Science and Technology,Shenzhen,518055,China
|First Author Affilication||Department of Computer Science and Engineering|
|Corresponding Author Affilication||Department of Computer Science and Engineering; Department of Statistics and Data Science|
|First Author's First Affilication||Department of Computer Science and Engineering|
Tang，Lan,Li，Xiaxi,Zhang，Jinyuan,et al. Enabling Surrogate-Assisted Evolutionary Reinforcement Learning via Policy Embedding[C],2023:233-247.
|Files in This Item:||There are no files associated with this item.|
|Recommend this item|
|Export to Endnote|
|Export to Excel|
|Export to Csv|
|Similar articles in Google Scholar|
|Similar articles in Baidu Scholar|
|Similar articles in Bing Scholar|
Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.