Reinforcement-learning-based parameter optimization of a splitter plate downstream in cylinder wake with stability analyses

Wang, Chengyun1; Yu, Peng2,3; Huang, Haibo1

2023-08-22

10.1103/PhysRevFluids.8.083904

PHYSICAL REVIEW FLUIDS   Impact Factor & Quartile

2469-990X

In this research, we apply the single-step deep reinforcement learning (DRL) algorithm to optimize the spatial location and length of a downstream splitter plate in order to suppress vortex shedding behind a cylinder. This algorithm is rare in optimization problems as it differs significantly from the traditional decision-making models. After a case with only one optimization parameter (streamwise position) is validated, the study is then extended to more complex cases with additional parameters (lateral position and plate length) to explore the full potential of the DRL-based optimization algorithm in high-dimensional design spaces. The final results are well explained by analyzing the local stability of the wake flow. In this context, we incorporate techniques commonly used in the DRL field, such as adaptive learning rates and reward shaping, which crucially complements and extends the emerging single-step DRL algorithms. By balancing exploration and exploitation phases and embedding additional stability information obtained through dynamic mode decomposition, the resulting method not only achieves improved global convergence but also reduces the computation time for each fluid environment. The specifics and infeasibility of another type of single-step DRL algorithm are detailed in the Appendix, and the comparison with Bayesian optimization is discussed by benchmarking on the same optimization problem. These results provide insights into the potential of single-step DRL algorithms and suggest that, with further fine-tuning for specific problems, it may outperform existing advanced optimization methods.

SCI

English

Others

Natural Science Foundation of China (NSFC)["11972342","12172163"]

Physics

Physics, Fluids & Plasmas

WOS:001063165100003

AMER PHYSICAL SOC

Web of Science

1.Univ Sci & Technol China, Dept Modern Mech, Hefei 230026, Anhui, Peoples R China
2.Southern Univ Sci & Technol, Dept Mech & Aerosp Engn, Shenzhen 518055, Guangdong, Peoples R China
3.Southern Univ Sci & Technol, Ctr Complex Flows & Soft Matter Res, Shenzhen 518055, Guangdong, Peoples R China

Wang, Chengyun,Yu, Peng,Huang, Haibo. Reinforcement-learning-based parameter optimization of a splitter plate downstream in cylinder wake with stability analyses[J]. PHYSICAL REVIEW FLUIDS,2023,8(8).

Wang, Chengyun,Yu, Peng,&Huang, Haibo.(2023).Reinforcement-learning-based parameter optimization of a splitter plate downstream in cylinder wake with stability analyses.PHYSICAL REVIEW FLUIDS,8(8).

Wang, Chengyun,et al."Reinforcement-learning-based parameter optimization of a splitter plate downstream in cylinder wake with stability analyses".PHYSICAL REVIEW FLUIDS 8.8(2023).