Multiscale Modeling &Simulation, Volume 19, Issue 1, Page 344-373, January 2021.
In scientific machine learning, regression networks have been recently applied to approximate solution maps (e.g., the potential-ground state map of the Schrödinger equation). In this paper, we aim to reduce the generalization error without spending more time on generating training samples. However, to reduce the generalization error, the regression network needs to be fit on a large number of training samples (e.g., a collection of potential-ground state pairs). The training samples can be produced by running numerical solvers, which takes significant time in many applications. In this paper, we aim to reduce the generalization error without spending more time on generating training samples. Inspired by few-shot learning techniques, we develop the multilevel fine-tuning algorithm by introducing levels of training: we first train the regression network on samples generated at the coarsest grid and then successively fine-tune the network on samples generated at finer grids. Within the same amount of time, numerical solvers generate more samples on coarse grids than on fine grids. We demonstrate a significant reduction of generalization error in numerical experiments on challenging problems with oscillations, discontinuities, or rough coefficients. Further analysis can be conducted in the neural tangent kernel regime, and we provide practical estimators to the generalization error. The number of training samples at different levels can be optimized for the smallest estimated generalization error under the constraint of budget for training data. The optimized distribution of budget over levels provides practical guidance with theoretical insight as in the celebrated multilevel Monte Carlo algorithm.

中文翻译：

---

多级微调：在有限的训练数据预算下缩小解图近似中的泛化差距

多尺度建模与仿真，第 19 卷，第 1 期，第 344-373 页，2021 年 1 月。
在科学机器学习中，回归网络最近已应用于近似解图（例如，薛定谔方程的势-基态图）。在本文中，我们的目标是在不花费更多时间生成训练样本的情况下减少泛化误差。然而，为了减少泛化误差，回归网络需要适应大量的训练样本（例如，潜在-基态对的集合）。可以通过运行数值求解器来生成训练样本，这在许多应用程序中需要花费大量时间。在本文中，我们的目标是在不花费更多时间生成训练样本的情况下减少泛化误差。受小样本学习技术的启发，我们通过引入训练级别来开发多级微调算法：我们首先在最粗网格生成的样本上训练回归网络，然后在更细网格生成的样本上连续微调网络。在相同的时间内，数值求解器在粗网格上生成的样本比在细网格上生成的样本多。我们证明了在具有振荡、不连续性或粗糙系数的挑战性问题的数值实验中泛化误差的显着减少。可以在神经切线核机制中进行进一步分析，我们提供了泛化误差的实用估计器。在训练数据预算的约束下，可以优化不同级别的训练样本数量以获得最小的估计泛化误差。