Many-query problems – arising from, e.g., uncertainty quantification, Bayesian inversion, Bayesian optimal experimental design, and optimization under uncertainty – require numerous evaluations of a parameter-to-output map. These evaluations become prohibitive if this parametric map is high-dimensional and involves expensive solution of partial differential equations (PDEs). To tackle this challenge, we propose to construct surrogates for high-dimensional PDE-governed parametric maps in the form of derivative-informed projected neural networks (DIPNets) that parsimoniously capture the geometry and intrinsic low-dimensionality of these maps. Specifically, we compute Jacobians of these PDE-based maps, and project the high-dimensional parameters onto a low-dimensional derivative-informed active subspace; we also project the possibly high-dimensional outputs onto their principal subspace. This exploits the fact that many high-dimensional PDE-governed parametric maps can be well-approximated in low-dimensional parameter and output subspaces. We use the projection basis vectors in the active subspace as well as the principal output subspace to construct the weights for the first and last layers of the neural network, respectively. This frees us to train the weights in only the low-dimensional layers of the neural network. The architecture of the resulting neural network then captures, to first order, the low-dimensional structure and geometry of the parametric map. We demonstrate that the proposed projected neural network achieves greater generalization accuracy than a full neural network, especially in the limited training data regime afforded by expensive PDE-based parametric maps. Moreover, we show that the number of degrees of freedom of the inner layers of the projected network is independent of the parameter and output dimensions, and high accuracy can be achieved with weight dimension independent of the discretization dimension.

多查询问题——例如不确定性量化、贝叶斯反演、贝叶斯最优实验设计和不确定性下的优化——需要对参数到输出映射进行大量评估。如果此参数映射是高维的并且涉及偏微分方程 (PDE) 的昂贵解，则这些评估变得令人望而却步。为了应对这一挑战，我们建议以导数信息投影神经网络 (DIPNets) 的形式为高维 PDE 控制的参数映射构建代理，该网络可简约地捕获这些映射的几何形状和内在低维。具体来说，我们计算这些基于 PDE 的映射的雅可比矩阵，并将高维参数投影到低维导数信息活动子空间；我们还将可能的高维输出投影到它们的主子空间上。这利用了许多高维 PDE 控制的参数映射可以很好地逼近低维参数和输出子空间的事实。我们使用活动子空间中的投影基向量以及主要输出子空间来分别构建神经网络第一层和最后一层的权重。这使我们能够仅在神经网络的低维层中训练权重。生成的神经网络的架构然后捕获参数映射的低维结构和几何形状。我们证明了所提出的投影神经网络比完整的神经网络实现了更高的泛化精度，特别是在昂贵的基于 PDE 的参数映射提供的有限训练数据机制中。此外，我们表明投影网络内层的自由度数与参数和输出维度无关，并且权重维度与离散化维度无关，可以实现高精度。