The unprecedented success of deep learning (DL) makes it unchallenged when it comes to classification problems. However, it is well established that the current DL methodology produces universally unstable neural networks (NNs). The instability problem has caused an enormous research effort -- with a vast literature on so-called adversarial attacks -- yet there has been no solution to the problem. Our paper addresses why there has been no solution to the problem, as we prove the following mathematical paradox: any training procedure based on training neural networks for classification problems with a fixed architecture will yield neural networks that are either inaccurate or unstable (if accurate) -- despite the provable existence of both accurate and stable neural networks for the same classification problems. The key is that the stable and accurate neural networks must have variable dimensions depending on the input, in particular, variable dimensions is a necessary condition for stability. Our result points towards the paradox that accurate and stable neural networks exist, however, modern algorithms do not compute them. This yields the question: if the existence of neural networks with desirable properties can be proven, can one also find algorithms that compute them? There are cases in mathematics where provable existence implies computability, but will this be the case for neural networks? The contrary is true, as we demonstrate how neural networks can provably exist as approximate minimisers to standard optimisation problems with standard cost functions, however, no randomised algorithm can compute them with probability better than 1/2.

中文翻译：

---

AI中对抗性攻击的数学——尽管存在稳定的神经网络，但深度学习为何不稳定

深度学习 (DL) 的空前成功使其在分类问题方面没有受到挑战。然而，众所周知，当前的 DL 方法会产生普遍不稳定的神经网络 (NN)。不稳定性问题引起了巨大的研究努力——有大量关于所谓对抗性攻击的文献——但还没有解决这个问题的方法。我们的论文解决了为什么没有解决这个问题的原因，因为我们证明了以下数学悖论：任何基于训练神经网络的训练过程，用于具有固定架构的分类问题都会产生不准确或不稳定的神经网络（如果准确的话） ——尽管可证明存在针对相同分类问题的准确和稳定的神经网络。关键是稳定且准确的神经网络必须具有取决于输入的可变维度，尤其是可变维度是稳定性的必要条件。我们的结果指向了一个悖论，即存在准确且稳定的神经网络，但是，现代算法并不计算它们。这就产生了一个问题：如果可以证明具有理想特性的神经网络的存在，那么是否还能找到计算它们的算法？在数学中存在可证明的存在性意味着可计算性的情况，但神经网络会是这种情况吗？事实恰恰相反，因为我们展示了神经网络如何可以证明作为标准成本函数的标准优化问题的近似最小值存在，但是，没有随机算法可以以超过 1/2 的概率计算它们。