Exponential expressivity of ${\rm ReLU}^k$ neural networks on Gevrey classes with point singularities
Metadata only
Date
2024-03Type
- Report
ETH Bibliography
yes
Altmetrics
Abstract
We analyze deep Neural Network emulation rates of smooth functions with point singularities in bounded, polytopal domains \(\mathrm{D} \subset \mathbb{R}^d\), \(d=2,3\). We prove exponential emulation rates in Sobolev spaces in terms of the number of neurons and in terms of the number of nonzero coefficients for Gevrey-regular solution classes defined in terms of weighted Sobolev scales in \(\mathrm{D}\), comprising the countably-normed spaces of I.M. Babu\v{s}ka and B.Q. Guo. As intermediate result, we prove that continuous, piecewise polynomial high order (``\(p\)-version'') finite elements with elementwise polynomial degree \(p\in\mathbb{N}\) on arbitrary, regular, simplicial partitions of polyhedral domains \(\mathrm{D} \subset \mathbb{R}^d\), \(d\geq 2\) can be \emph{exactly emulated} by neural networks combining ReLU and ReLU\(^2\) activations. On shape-regular, simplicial partitions of polytopal domains \(\mathrm{D}\), both the number of neurons and the number of nonzero parameters are proportional to the number of degrees of freedom of the finite element space, in particular for the \(hp\)-Finite Element Method of I.M. Babu\v{s}ka and B.Q. Guo. Show more
Publication status
publishedExternal links
Journal / series
SAM Research ReportVolume
Publisher
Seminar for Applied Mathematics, ETH ZurichSubject
Neural networks; hp-finite element method; Singularities; Gevrey regularity; Exponential convergenceOrganisational unit
03435 - Schwab, Christoph / Schwab, Christoph
Related publications and datasets
Is previous version of: https://doi.org/10.3929/ethz-b-000691409
More
Show all metadata
ETH Bibliography
yes
Altmetrics