Deep neural networks (DNNs) have surpassed human-level accuracy in various learning tasks. However, unlike humans who have a natural cognitive intuition for probabilities, DNNs cannot express their uncertainty in the output decisions. This limits the deployment of DNNs in mission-critical domains, such as warfighter decision-making or medical diagnosis. Bayesian inference provides a principled approach to reason about model's uncertainty by estimating the posterior distribution of the unknown parameters. The challenge in DNNs remains the multi-layer stages of non-linearities, which make the propagation of high-dimensional distributions mathematically intractable. This paper establishes the theoretical and algorithmic foundations of uncertainty or belief propagation by developing new deep learning models named PremiUm-CNNs (Propagating Uncertainty in Convolutional Neural Networks). We introduce a tensor normal distribution as a prior over convolutional kernels and estimate the variational posterior by maximizing the evidence lower bound (ELBO). We start by deriving the first-order mean-covariance propagation framework. Later, we develop a framework based on the unscented transformation (correct at least up to the second-order) that propagates sigma points of the variational distribution through layers of a CNN. The propagated covariance of the predictive distribution captures uncertainty in the output decision. Comprehensive experiments conducted on diverse benchmark datasets demonstrate: 1) superior robustness against noise and adversarial attacks, 2) self-assessment through predictive uncertainty that increases quickly with increasing levels of noise or attacks, and 3) an ability to detect a targeted attack from ambient noise.

中文翻译：

---

PremiUm-CNN：向鲁棒卷积神经网络传播不确定性


深度神经网络（DNN）在各种学习任务中的准确性已经超过了人类水平。然而，与人类对概率具有自然的认知直觉不同，DNN 无法表达输出决策中的不确定性。这限制了 DNN 在关键任务领域的部署，例如作战决策或医疗诊断。贝叶斯推理提供了一种通过估计未知参数的后验分布来推理模型不确定性的原理方法。 DNN 的挑战仍然是非线性的多层阶段，这使得高维分布的传播在数学上难以处理。本文通过开发名为 PremiUm-CNN（卷积神经网络中的传播不确定性）的新型深度学习模型，建立了不确定性或信念传播的理论和算法基础。我们引入张量正态分布作为卷积核的先验，并通过最大化证据下界（ELBO）来估计变分后验。我们首先推导一阶均值协方差传播框架。后来，我们开发了一个基于无味变换（至少正确到二阶）的框架，该框架通过 CNN 层传播变分分布的西格玛点。预测分布的传播协方差捕获了输出决策中的不确定性。对不同基准数据集进行的综合实验表明：1）针对噪声和对抗性攻击具有卓越的鲁棒性，2）通过预测不确定性进行自我评估，这种不确定性随着噪声或攻击水平的增加而迅速增加，3）从环境中检测有针对性的攻击的能力噪音。