Improving the accuracy and robustness of deep neural nets (DNNs) and adapting them to small training data are primary tasks in deep learning (DL) research. In this paper, we replace the output activation function of DNNs, typically the data-agnostic softmax function, with a graph Laplacian-based high-dimensional interpolating function which, in the continuum limit, converges to the solution of a Laplace–Beltrami equation on a high-dimensional manifold. Furthermore, we propose end-to-end training and testing algorithms for this new architecture. The proposed DNN with graph interpolating activation integrates the advantages of both deep learning and manifold learning. Compared to the conventional DNNs with the softmax function as output activation, the new framework demonstrates the following major advantages: First, it is better applicable to data-efficient learning in which we train high capacity DNNs without using a large number of training data. Second, it remarkably improves both natural accuracy on the clean images and robust accuracy on the adversarial images crafted by both white-box and black-box adversarial attacks. Third, it is a natural choice for semi-supervised learning. This paper is a significant extension of our earlier work published in NeurIPS, 2018. For reproducibility, the code is available at https://github.com/BaoWangMath/DNN-DataDependentActivation.

中文翻译：

---

图插值激活提高了数据高效深度学习的自然准确度和鲁棒准确度

提高深度神经网络 (DNN) 的准确性和鲁棒性并使其适应小型训练数据是深度学习 (DL) 研究的主要任务。在本文中，我们将 DNN 的输出激活函数（通常是与数据无关的 softmax 函数）替换为基于图拉普拉斯算子的高维插值函数，该函数在连续极限下收敛到一个高维流形。此外，我们为这种新架构提出了端到端的训练和测试算法。所提出的具有图插值激活的 DNN 集成了深度学习和流形学习的优点。与以 softmax 函数作为输出激活的传统 DNN 相比，新框架具有以下主要优势：它更适用于数据高效学习，我们在不使用大量训练数据的情况下训练高容量 DNN。其次，它显着提高了干净图像的自然准确性和白盒和黑盒对抗性攻击制作的对抗性图像的鲁棒准确性。第三，它是半监督学习的自然选择。本文是我们在 2018 年 NeurIPS 上发表的早期工作的重要扩展。为了可重复性，代码可在https://github.com/BaoWangMath/DNN-DataDependentActivation.