Bayesian neural Networks (BNNs) are a promising method of obtaining statistical uncertainties for neural network predictions but with a higher computational overhead which can limit their practical usage. This work explores the use of high performance computing with distributed training to address the challenges of training BNNs at scale. We present a performance and scalability comparison of training the VGG-16 and Resnet-18 models on a Cray-XC40 cluster. We demonstrate that network pruning can speed up inference without accuracy loss and provide an open source software package, {\it{BPrune}} to automate this pruning. For certain models we find that pruning up to 80\% of the network results in only a 7.0\% loss in accuracy. With the development of new hardware accelerators for Deep Learning, BNNs are of considerable interest for benchmarking performance. This analysis of training a BNN at scale outlines the limitations and benefits compared to a conventional neural network.

中文翻译：

---

大规模贝叶斯神经网络：性能分析和剪枝研究

贝叶斯神经网络 (BNN) 是一种获得神经网络预测统计不确定性的有前途的方法，但其计算开销较高，这可能会限制其实际使用。这项工作探索了使用高性能计算和分布式训练来解决大规模训练 BNN 的挑战。我们展示了在 Cray-XC40 集群上训练 VGG-16 和 Resnet-18 模型的性能和可扩展性比较。我们证明网络修剪可以在不损失准确性的情况下加快推理速度，并提供一个开源软件包 {\it{BPrune}} 来自动进行修剪。对于某些模型，我们发现修剪多达 80% 的网络只会导致精度损失 7.0%。随着用于深度学习的新硬件加速器的开发，BNN 对基准性能非常感兴趣。这种大规模训练 BNN 的分析概述了与传统神经网络相比的局限性和好处。