当前位置: X-MOL 学术Neural Netw. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Uncertainty propagation for dropout-based Bayesian neural networks
Neural Networks ( IF 7.8 ) Pub Date : 2021-09-09 , DOI: 10.1016/j.neunet.2021.09.005
Yuki Mae 1 , Wataru Kumagai 2 , Takafumi Kanamori 3
Affiliation  

Uncertainty evaluation is a core technique when deep neural networks (DNNs) are used in real-world problems. In practical applications, we often encounter unexpected samples that have not seen in the training process. Not only achieving the high-prediction accuracy but also detecting uncertain data is significant for safety-critical systems. In statistics and machine learning, Bayesian inference has been exploited for uncertainty evaluation. The Bayesian neural networks (BNNs) have recently attracted considerable attention in this context, as the DNN trained using dropout is interpreted as a Bayesian method. Based on this interpretation, several methods to calculate the Bayes predictive distribution for DNNs have been developed. Though the Monte-Carlo method called MC dropout is a popular method for uncertainty evaluation, it requires a number of repeated feed-forward calculations of DNNs with randomly sampled weight parameters. To overcome the computational issue, we propose a sampling-free method to evaluate uncertainty. Our method converts a neural network trained using dropout to the corresponding Bayesian neural network with variance propagation. Our method is available not only to feed-forward NNs but also to recurrent NNs such as LSTM. We report the computational efficiency and statistical reliability of our method in numerical experiments of language modeling using RNNs, and the out-of-distribution detection with DNNs.



中文翻译:

基于 dropout 的贝叶斯神经网络的不确定性传播

当深度神经网络 (DNN) 用于解决实际问题时,不确定性评估是一项核心技术。在实际应用中,我们经常会遇到训练过程中没有见过的意外样本。不仅要实现高预测精度,而且检测不确定数据对于安全关键系统也很重要。在统计学和机器学习中,贝叶斯推理已被用于不确定性评估。贝叶斯神经网络 (BNN) 最近在这方面引起了相当大的关注,因为使用 dropout 训练的 DNN 被解释为贝叶斯方法。基于这种解释,已经开发了几种计算 DNN 的贝叶斯预测分布的方法。尽管称为 MC dropout 的 Monte-Carlo 方法是一种流行的不确定性评估方法,它需要使用随机采样的权重参数对 DNN 进行多次重复的前馈计算。为了克服计算问题,我们提出了一种无采样方法来评估不确定性。我们的方法将使用 dropout 训练的神经网络转换为相应的贝叶斯神经网络方差传播。我们的方法不仅适用于前馈神经网络,还适用于循环神经网络,如 LSTM。我们报告了我们的方法在使用 RNN 进行语言建模的数值实验中的计算效率和统计可靠性,以及使用 DNN 进行的分布外检测。

更新日期:2021-09-23
down
wechat
bug