Exponential ReLU Neural Network Approximation Rates for Point and Edge Singularities
Metadata only
Date
2020-10Type
- Report
ETH Bibliography
yes
Altmetrics
Abstract
We prove exponential expressivity with stable ReLU Neural Networks (ReLU NNs) in H1(Ω) for weighted analytic function classes in certain polytopal domains Ω, in space dimension d=2,3. Functions in these classes are locally analytic on open subdomains D⊂Ω, but may exhibit isolated point singularities in the interior of Ω or corner and edge singularities at the boundary ∂Ω. The exponential expression rate bounds proved here imply uniform exponential expressivity by ReLU NNs of solution families for several elliptic boundary and eigenvalue problems with analytic data. The exponential approximation rates are shown to hold in space dimension d=2 on Lipschitz polygons with straight sides, and in space dimension d=3 on Fichera-type polyhedral domains with plane faces. The constructive proofs indicate in particular that NN depth and size increase poly-logarithmically with respect to the target NN approximation accuracy ε>0 in H1(Ω). The results cover in particular solution sets of linear, second order elliptic PDEs with analytic data and certain nonlinear elliptic eigenvalue problems with analytic nonlinearities and singular, weighted analytic potentials as arise in electron structure models. In the latter case, the functions correspond to electron densities that exhibit isolated point singularities at the positions of the nuclei. Our findings provide in particular mathematical foundation of recently reported, successful uses of deep neural networks in variational electron structure algorithms. Show more
Publication status
publishedExternal links
Journal / series
SAM Research ReportVolume
Publisher
Seminar for Applied Mathematics, ETH ZurichSubject
Neural networks; finite element methods; exponential convergence; analytic regularity; singularitiesOrganisational unit
03435 - Schwab, Christoph / Schwab, Christoph
Related publications and datasets
Is previous version of: https://doi.org/10.3929/ethz-b-000554389
More
Show all metadata
ETH Bibliography
yes
Altmetrics