中文版 | English
Title

Enabling surrogate-assisted evolutionary reinforcement learning via policy embedding

Author
Corresponding AuthorPeng Yang
Publication Years
2022-12
Conference Name
the 17th International Conference on Bio-inspired Computing: Theories and Applications (BIC-TA 2022)
Source Title
Conference Date
2022-12-16
Conference Place
武汉
Abstract

Evolutionary Reinforcement Learning (ERL) that applying Evolutionary Algorithms (EAs) to optimize the weight parameters of Deep Neural Network (DNN) based policies has been widely regarded as an alternative to traditional reinforcement learning methods. However, the evaluation of the iteratively generated population usually requires a large amount of computational time and can be prohibitively expensive, which may potentially restrict the applicability of ERL. Surrogate is often used to reduce the computational burden of evaluation in EAs. Unfortunately, in ERL, each individual of policy usually represents millions of weights parameters of DNN. This high-dimensional representation of policy has introduced a great challenge to the application of surrogates into ERL to speed up training. This paper proposes a PE-SAERL Framework to at the first time enable surrogate-assisted evolutionary reinforcement learning via policy embedding (PE). Empirical results on 5 Atari games show that the proposed method can perform more efficiently than the four state-of-the-art algorithms. The training process is accelerated up to 7x on tested games, comparing to its counterpart without the surrogate and PE.

Keywords
SUSTech Authorship
First ; Corresponding
Language
English
Indexed By
Data Source
人工提交
Document TypeConference paper
Identifierhttp://kc.sustech.edu.cn/handle/2SGJ60CL/523892
DepartmentDepartment of Computer Science and Engineering
理学院_统计与数据科学系
工学院_斯发基斯可信自主研究院
Affiliation
1.Guangdong Provincial Key Laboratory of Brain-Inspired Intelligent Computation, Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen 518055, Chin
2.Faculty of engineering, Shenzhen MSU-BIT university, Shenzhen 518172, Chin
3.Research Institute of Trustworthy Autonomous Systems, Southern University of Science and Technology, Shenzhen 518055, Chin
4.Department of Statistics and Data Science, Southern University of Science and Technology, Shenzhen 518055, Chin
First Author AffilicationDepartment of Computer Science and Engineering
Corresponding Author AffilicationDepartment of Computer Science and Engineering;  Department of Statistics and Data Science
First Author's First AffilicationDepartment of Computer Science and Engineering
Recommended Citation
GB/T 7714
Lan Tang,Xiaxi Li,Jinyuan Zhang,et al. Enabling surrogate-assisted evolutionary reinforcement learning via policy embedding[C],2022.
Files in This Item:
There are no files associated with this item.
Related Services
Fulltext link
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Export to Excel
Export to Csv
Altmetrics Score
Google Scholar
Similar articles in Google Scholar
[Lan Tang]'s Articles
[Xiaxi Li]'s Articles
[Jinyuan Zhang]'s Articles
Baidu Scholar
Similar articles in Baidu Scholar
[Lan Tang]'s Articles
[Xiaxi Li]'s Articles
[Jinyuan Zhang]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Lan Tang]'s Articles
[Xiaxi Li]'s Articles
[Jinyuan Zhang]'s Articles
Terms of Use
No data!
Social Bookmark/Share
No comment.

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.