Linear attention coupled Fourier neural operator for simulation of three-dimensional turbulence

Peng, Wenhui1,2,3; Yuan, Zelong1,2; Li, Zhijie1,2; Wang, Jianchun1,2

2023

10.1063/5.0130334

PHYSICS OF FLUIDS   Impact Factor & Quartile

1070-6631

1089-7666

Modeling three-dimensional (3D) turbulence by neural networks is difficult because 3D turbulence is highly nonlinear with high degrees of freedom and the corresponding simulation is memory-intensive. Recently, the attention mechanism has been shown as a promising approach to boost the performance of neural networks on turbulence simulation. However, the standard self-attention mechanism uses O ( n (2) ) time and space with respect to input dimension n, and such quadratic complexity has become the main bottleneck for attention to be applied on 3D turbulence simulation. In this work, we resolve this issue with the concept of a linear attention network. The linear attention approximates the standard attention by adding two linear projections, reducing the overall self-attention complexity from O ( n( 2) ) to O(n) in both time and space. The linear attention coupled Fourier neural operator (LAFNO) is developed for the simulation of 3D isotropic turbulence and free shear turbulence. Numerical simulations show that the linear attention mechanism provides 40% error reduction at the same level of computational cost, and LAFNO can accurately reconstruct a variety of statistics and instantaneous spatial structures of 3D turbulence. The linear attention method would be helpful for the improvement of neural network models of 3D nonlinear problems involving high-dimensional data in other scientific domains.

SCI

English

First ; Corresponding

National Natural Science Foundation of China (NSFC)["91952104","92052301","12172161","91752201"] ; National Numerical Wind Tunnel Project[NNW2019ZT1-A04] ; Shenzhen Science and Technology Program[KQTD20180411143441009] ; Key Special Project for Introduced Talents Team of Southern Marine Science and Engineering Guangdong Laboratory (Guangzhou)[GML2019ZD0103] ; Department of Science and Technology of Guangdong Province[2020B1212030001]

Mechanics ; Physics

Mechanics ; Physics, Fluids & Plasmas

WOS:000912374800001

AIP Publishing

PHYSICS

Web of Science

1.Southern Univ Sci & Technol, Dept Mech & Aerosp Engn, Shenzhen 518055, Peoples R China
2.Southern Univ Sci & Technol, Guangdong Hong Kong Macao Joint Lab Data Driven Fl, Hong Kong 518055, Guangdong, Peoples R China
3.Polytech Montreal, Dept Comp Engn, Quebec City, PQ H3T 1J4, Canada

Peng, Wenhui,Yuan, Zelong,Li, Zhijie,et al. Linear attention coupled Fourier neural operator for simulation of three-dimensional turbulence[J]. PHYSICS OF FLUIDS,2023,35(1).

Peng, Wenhui,Yuan, Zelong,Li, Zhijie,&Wang, Jianchun.(2023).Linear attention coupled Fourier neural operator for simulation of three-dimensional turbulence.PHYSICS OF FLUIDS,35(1).

Peng, Wenhui,et al."Linear attention coupled Fourier neural operator for simulation of three-dimensional turbulence".PHYSICS OF FLUIDS 35.1(2023).