Exponential expressivity of ${\rm ReLU}^k$ neural networks on Gevrey classes with point singularities
METADATA ONLY
Loading...
Author / Producer
Date
2024-03
Publication Type
Report
ETH Bibliography
yes
Citations
Altmetric
METADATA ONLY
Data
Rights / License
Abstract
We analyze deep Neural Network emulation rates of smooth functions with point singularities in bounded, polytopal domains \(\mathrm{D} \subset \mathbb{R}^d\), \(d=2,3\). We prove exponential emulation rates in Sobolev spaces in terms of the number of neurons and in terms of the number of nonzero coefficients for Gevrey-regular solution classes defined in terms of weighted Sobolev scales in \(\mathrm{D}\), comprising the countably-normed spaces of I.M. Babu\v{s}ka and B.Q. Guo. As intermediate result, we prove that continuous, piecewise polynomial high order (``\(p\)-version'') finite elements with elementwise polynomial degree \(p\in\mathbb{N}\) on arbitrary, regular, simplicial partitions of polyhedral domains \(\mathrm{D} \subset \mathbb{R}^d\), \(d\geq 2\) can be \emph{exactly emulated} by neural networks combining ReLU and ReLU\(^2\) activations. On shape-regular, simplicial partitions of polytopal domains \(\mathrm{D}\), both the number of neurons and the number of nonzero parameters are proportional to the number of degrees of freedom of the finite element space, in particular for the \(hp\)-Finite Element Method of I.M. Babu\v{s}ka and B.Q. Guo.
Permanent link
Publication status
published
Editor
Book title
Journal / series
Volume
2024-11
Pages / Article No.
Publisher
Seminar for Applied Mathematics, ETH Zurich
Event
Edition / version
Methods
Software
Geographic location
Date collected
Date created
Subject
Neural networks; hp-finite element method; Singularities; Gevrey regularity; Exponential convergence
Organisational unit
03435 - Schwab, Christoph / Schwab, Christoph
Notes
Funding
Related publications and datasets
Is previous version of: