Search
Results
-
An operator preconditioning perspective on training in physics-informed machine learning
(2023)SAM Research ReportIn this paper, we investigate the behavior of gradient descent algorithms in physics-informed machine learning methods like PINNs, which minimize resid uals connected to partial differential equations (PDEs). Our key result is that the difficulty in training these models is closely related to the conditioning of a specific differential operator. This operator, in turn, is associated to the Hermi tian square of the differential operator ...Report -
Multi-Scale Message Passing Neural PDE Solvers
(2023)SAM Research ReportWe propose a novel multi-scale message passing neural network algorithm for learning the solutions of time-dependent PDEs. Our algorithm possesses both temporal and spatial multi-scale resolution features by incorporating multi-scale sequence models and graph gating modules in the encoder and processor, respectively. Benchmark numerical experiments are presented to demonstrate that the proposed algorithm outperforms baselines, particularly ...Report -
A Monte-Carlo ab-initio algorithm for the multiscale simulation of compressible multiphase flows
(2023)SAM Research ReportWe propose a novel Monte-Carlo based ab-initio algorithm for directly computing the statistics for quantities of interest in an immiscible two-phase compressible flow. Our algorithm samples the underlying probability space and evolves these samples with a sharp interface front-tracking scheme. Consequently, statistical information is generated without resorting to any closure assumptions and information about the underlying microstructure ...Report -
A Survey on Oversmoothing in Graph Neural Networks
(2023)SAM Research ReportNode features of graph neural networks (GNNs) tend to become more similar with the increase of the network depth. This effect is known as over-smoothing, which we axiomatically define as the exponential convergence of suitable similarity measures on the node features. Our definition unifies previous approaches and gives rise to new quantitative measures of over-smoothing. Moreover, we empirically demonstrate this behavior for several ...Report -
Neural Inverse Operators for Solving PDE Inverse Problems
(2023)SAM Research ReportA large class of inverse problems for PDEs are only well-defined as mappings from operators to functions. Existing operator learning frameworks map functions to functions and need to be modified to learn inverse maps from data. We propose a novel architecture termed Neural Inverse Operators (NIOs) to solve these PDE inverse problems. Motivated by the underlying mathematical structure, NIO is based on a suitable composition of DeepONets ...Report -
Convolutional Neural Operators
(2023)SAM Research ReportAlthough very successfully used in machine learning, convolution based neural network architectures -- believed to be inconsistent in function space -- have been largely ignored in the context of learning solution operators of PDEs. Here, we adapt convolutional neural networks to demonstrate that they are indeed able to process functions as inputs and outputs. The resulting architecture, termed as convolutional neural operators (CNOs), ...Report -
Are Neural Operators Really Neural Operators? Frame Theory Meets Operator Learning
(2023)SAM Research ReportRecently, there has been significant interest in operator learning, i.e. learning mappings between infinite-dimensional function spaces. This has been particularly relevant in the context of learning partial differential equations from data. However, it has been observed that proposed models may not behave as operators when implemented on a computer, questioning the very essence of what operator learning should be. We contend that in ...Report -
Multilevel domain decomposition-based architectures for physics-informed neural networks
(2023)SAM Research ReportPhysics-informed neural networks (PINNs) are a popular and powerful approach for solving problems involving differential equations, yet they often struggle to solve problems with high frequency and/or multi-scale solutions. Finite basis physics-informed neural networks (FBPINNs) improve the performance of PINNs in this regime by combining them with an overlapping domain decomposition approach. In this paper, the FBPINN approach is extended ...Report -
Convolutional Neural Operators for robust and accurate learning of PDEs
(2023)SAM Research ReportAlthough very successfully used in conventional machine learning, convolution based neural network architectures -- believed to be inconsistent in function space -- have been largely ignored in the context of learning solution operators of PDEs. Here, we present novel adaptations for convolutional neural networks to demonstrate that they are indeed able to process functions as inputs and outputs. The resulting architecture, termed as ...Report -
Neural Oscillators are Universal
(2023)SAM Research ReportCoupled oscillators are being increasingly used as the basis of machine learning (ML) architectures, for instance in sequence modeling, graph representation learning and in physical neural networks that are used in analog ML devices. We introduce an abstract class of neural oscillators that encompasses these architectures and prove that neural oscillators are universal, i.e, they can approximate any continuous and casual operator mapping ...Report