Deep learning in high dimension: ReLU network Expression Rates for Bayesian PDE inversion

Abstract

We establish dimension independent expression rates by deep ReLU networks for so-called (b,ε,X)-holomorphic functions. These are mappings from [−1,1]N→X, with X being a Banach space, that admit analytic extensions to certain polyellipses in each of the input variables. The significance of this function class has been established in previous works, where it was shown that functions of this type occur widely in uncertainty quantification for partial differential equations with uncertain inputs from function spaces. Proofs for establishing the expression rate bounds are constructive, and are based on multilevel polynomial chaos expansions of the target function. The (b,ε,X)-holomorphy facilitates estimation of the coefficients in the polynomial chaos expansions. We apply the results to Bayesian inverse problems for partial differential equations with distributed, uncertain inputs from Banach spaces, resulting in expression rate bounds on the Bayesian posterior densities by deep ReLU neural networks. The expression rates for these countably-parametric maps are free from the curse of dimensionality. Certain types of Bayesian posterior concentration, which generically arise in large data or small noise asymptotics (e.g. [B. T. Knapik and A. W. van der Vaart and J. H. van Zanten, 2011]) can be emulated in a noise-robust fashion by the ability of ReLU DNNs to express the geometry of possibly high-dimensional posterior densities at MAP points.