We propose a theoretical understanding of neural networks in terms of Wilsonian effective field theory. The correspondence relies on the fact that many asymptotic neural networks are drawn from Gaussian processes (GPs), the analog of non-interacting field theories. Moving away from the asymptotic limit yields a non-Gaussian process (NGP) and corresponds to turning on particle interactions, allowing for the computation of correlation functions of neural network outputs with Feynman diagrams. Minimal NGP likelihoods are determined by the most relevant non-Gaussian terms, according to the flow in their coefficients induced by the Wilsonian renormalization group. This yields a direct connection between overparameterization and simplicity of neural network likelihoods. Whether the coefficients are constants or functions may be understood in terms of GP limit symmetries, as expected from ’t Hooft’s technical naturalness. General theoretical calculations are matched to neural network experiments in the simplest class of models allowing the correspondence. Our formalism is valid for any of the many architectures that becomes a GP in an asymptotic limit, a property preserved under certain types of training.

我们根据威尔逊有效场论提出了对神经网络的理论理解。这种对应关系依赖于这样一个事实，即许多渐近神经网络都是从高斯过程 (GP) 中得出的，GPs 是非相互作用场论的类似物。远离渐近极限会产生非高斯过程 (NGP)，并对应于打开粒子相互作用，从而允许使用费曼图计算神经网络输出的相关函数。最小 NGP 似然性由最相关的非高斯项确定，根据威尔逊重整化组引起的系数中的流动。这产生了过度参数化和神经网络可能性的简单性之间的直接联系。正如 't Hooft 的技术自然性所预期的那样，系数是常数还是函数都可以根据 GP 极限对称性来理解。一般理论计算与允许对应的最简单模型中的神经网络实验相匹配。我们的形式主义适用于在渐近极限中成为 GP 的许多架构中的任何一个，这是在某些类型的训练下保留的属性。