We propose quadratic residual networks (QRes) as a new type of parameter-efficient neural network architecture, by adding a quadratic residual term to the weighted sum of inputs before applying activation functions. With sufficiently high functional capacity (or expressive power), we show that it is especially good for solving forward and inverse physics problems involving partial differential equations (PDEs). Using tools from algebraic geometry, we theoretically demonstrate that, in contrast to plain neural networks, QRes shows better parameter efficiency in terms of network width and depth thanks to higher non-linearity in every neuron. Finally, we empirically show that QRes shows faster convergence speed in terms of number of training epochs especially in learning complex patterns.

中文翻译：

---

二次残数网络：一类新的神经网络，用于解决涉及PDE的物理学中的正反问题

我们提出二次残差网络（QRes）作为一种新型的参数有效神经网络架构，方法是在应用激活函数之前，将二次残差项添加到输入的加权总和中。具有足够高的功能能力（或表达能力），我们证明它特别适用于解决涉及偏微分方程（PDE）的正向和逆向物理问题。理论上，我们使用代数几何中的工具证明，与普通神经网络相比，QRes由于每个神经元的非线性程度更高，因此在网络宽度和深度方面显示出更好的参数效率。最后，我们从经验上证明，QRes在训练时期数方面表现出更快的收敛速度，尤其是在学习复杂模式方面。