Multi-Objective Loss Balancing for Physics-Informed Deep Learning
OPEN ACCESS
Loading...
Author / Producer
Date
2025-05
Publication Type
Journal Article, Journal Article
ETH Bibliography
yes
Citations
Altmetric
OPEN ACCESS
Data
Rights / License
Abstract
Physics-Informed Neural Networks (PINN) are deep learning algorithms that leverage physical laws by including partial differential equations together with a respective set of boundary and initial conditions as penalty terms in their loss function. In this work, we observe the significant role of correctly weighting the combination of multiple competitive loss functions for training PINNs effectively. To this end, we implement and evaluate different methods aiming at balancing the contributions of multiple terms of the PINN’s loss function and their gradients. After reviewing three existing loss scaling approaches (Learning Rate Annealing, GradNorm and SoftAdapt), we propose a novel self-adaptive loss balancing scheme for PINNs named ReLoBRaLo (Relative Loss Balancing with Random Lookback). We extensively evaluate the performance of the aforementioned balancing schemes by solving both forward as well as inverse problems on three benchmark PDEs for PINNs: Burgers’ equation, Kirchhoff’s plate bending equation, Helmholtz’s equation and over 20 PDEs from the ”PINNacle” collection. The results show that ReLoBRaLo is able to consistently outperform the baseline of existing scaling methods in terms of accuracy while also inducing significantly less computational overhead for a variety of PDE classes.
Permanent link
Publication status
published
External links
Editor
Book title
Journal / series
Volume
439
Pages / Article No.
117914
Publisher
Elsevier
Event
Edition / version
Methods
Software
Geographic location
Date collected
Date created
Subject
Machine Learning; Physics informed neural networks (PINNs); Multi objective optimization; Partial differential equations; Scientific machine learning; Helmholtz equation; Burgers equation; PINNacle; Loss balancing; Multi-objective optimisation
Organisational unit
09809 - Bickel, Bernd / Bickel, Bernd