Differentiable modelling to unify machine learning and physical models for geosciences

Shen，Chaopeng1; Appling，Alison P.2; Gentine，Pierre3; Bandai，Toshiyuki4; Gupta，Hoshin5; Tartakovsky，Alexandre6; Baity-Jesi，Marco7; Fenicia，Fabrizio7; Kifer，Daniel8; Li，Li1; Liu，Xiaofeng1; Ren，Wei9; Zheng，Yi10; Harman，Ciaran J.11; Clark，Martyn12; Farthing，Matthew13; Feng，Dapeng1; Kumar，Praveen6,14; Aboelyazeed，Doaa1; Rahmani，Farshid1; Song，Yalan1; Beck，Hylke E.15; Bindas，Tadd1; Dwivedi，Dipankar16; Fang，Kuai17; Höge，Marvin7; Rackauckas，Chris18; Mohanty，Binayak19; Roy，Tirthankar20; Xu，Chonggang21; Lawson，Kathryn1

2023-08-01

10.1038/s43017-023-00450-9

Nature Reviews Earth and Environment   Impact Factor & Quartile

2662-138X

Process-based modelling offers interpretability and physical consistency in many domains of geosciences but struggles to leverage large datasets efficiently. Machine-learning methods, especially deep networks, have strong predictive skills yet are unable to answer specific scientific questions. In this Perspective, we explore differentiable modelling as a pathway to dissolve the perceived barrier between process-based modelling and machine learning in the geosciences and demonstrate its potential with examples from hydrological modelling. ‘Differentiable’ refers to accurately and efficiently calculating gradients with respect to model variables or parameters, enabling the discovery of high-dimensional unknown relationships. Differentiable modelling involves connecting (flexible amounts of) prior physical knowledge to neural networks, pushing the boundary of physics-informed machine learning. It offers better interpretability, generalizability, and extrapolation capabilities than purely data-driven machine learning, achieving a similar level of accuracy while requiring less training data. Additionally, the performance and efficiency of differentiable models scale well with increasing data volumes. Under data-scarce scenarios, differentiable models have outperformed machine-learning models in producing short-term dynamics and decadal-scale trends owing to the imposed physical constraints. Differentiable modelling approaches are primed to enable geoscientists to ask questions, test hypotheses, and discover unrecognized physical relationships. Future work should address computational challenges, reduce uncertainty, and verify the physical significance of outputs.

SCI

English

Others

National Science Foundation EAR[2015680] ; National Science Foundation[EAR-2221880] ; Office of Science, US Department of Energy[DE-SC0016605] ; Cooperative Institute for Research to Operations in Hydrology (CIROH)[A22-0307-S003] ; National Science Foundational Science and Technology Center, Learning the Earth with Artificial intelligence and Physics (LEAP)[2019625]

Environmental Sciences & Ecology ; Geology

Environmental Sciences ; Geosciences, Multidisciplinary

WOS:001026496700001

SPRINGERNATURE

2-s2.0-85164665046

Scopus

1.Civil and Environmental Engineering,The Pennsylvania State University,University Park,United States
2.US Geological Survey,Reston,United States
3.National Science Foundation Science and Technology Center for Learning the Earth with Artificial Intelligence and Physics (LEAP),Columbia University,New York,United States
4.Energy Geoscience Divisions,Earth and Environmental Sciences Area,Lawrence Berkeley National Laboratory,Berkeley,United States
5.Hydrology and Atmospheric Sciences,The University of Arizona,Tucson,United States
6.Civil and Environmental Engineering,University of Illinois,Urbana Champaign,United States
7.Eawag: Swiss Federal Institute of Aquatic Science and Technology,Dübendorf,Switzerland
8.Computer Science and Engineering,The Pennsylvania State University,University Park,United States
9.Department of Natural Resources and the Environment,University of Connecticut,Storrs,United States
10.Southern University of Science and Technology,Shenzhen,Guangdong Province,China
11.Department of Environmental Health and Engineering,Johns Hopkins University,Baltimore,United States
12.Global Institute for Water Security,University of Saskatchewan,Canmore,Canada
13.US Army Engineer Research and Development Center,Vicksburg,United States
14.Prairie Research Institute,University of Illinois,Urbana Champaign,United States
15.Physical Science and Engineering Division,King Abdullah University of Science and Technology,Thuwal,Saudi Arabia
16.Climate and Ecosystem Sciences Divisions,Earth and Environmental Sciences Area,Lawrence Berkeley National Laboratory,Berkeley,United States
17.Department of Earth System Science,Stanford University,Stanford,United States
18.Computer Science and Artificial Intelligence Laboratory (CSAIL),Massachusetts Institute of Technology,Cambridge,United States
19.Department of Biological and Agricultural Engineering,Texas A&M University,College Station,United States
20.Civil and Environmental Engineering,University of Nebraska-Lincoln,Lincoln,United States
21.Earth and Environmental Sciences Division,Los Alamos National Laboratory,New Mexico,United States

Shen，Chaopeng,Appling，Alison P.,Gentine，Pierre,et al. Differentiable modelling to unify machine learning and physical models for geosciences[J]. Nature Reviews Earth and Environment,2023,4(8):552-567.

Shen，Chaopeng.,Appling，Alison P..,Gentine，Pierre.,Bandai，Toshiyuki.,Gupta，Hoshin.,...&Lawson，Kathryn.(2023).Differentiable modelling to unify machine learning and physical models for geosciences.Nature Reviews Earth and Environment,4(8),552-567.

Shen，Chaopeng,et al."Differentiable modelling to unify machine learning and physical models for geosciences".Nature Reviews Earth and Environment 4.8(2023):552-567.