Abstract Deep feedforward neural networks (DFNNs) are a powerful tool for functional approximation. We describe flexible versions of generalized linear and generalized linear mixed models incorporating basis functions formed by a DFNN. The consideration of neural networks with random effects is not widely used in the literature, perhaps because of the computational challenges of incorporating subject specific parameters into already complex models. Efficient computational methods for high-dimensional Bayesian inference are developed using Gaussian variational approximation, with a parsimonious but flexible factor parameterization of the covariance matrix. We implement natural gradient methods for the optimization, exploiting the factor structure of the variational covariance matrix in computation of the natural gradient. Our flexible DFNN models and Bayesian inference approach lead to a regression and classification method that has a high prediction accuracy, and is able to quantify the prediction uncertainty in a principled and convenient way. We also describe how to perform variable selection in our deep learning method. The proposed methods are illustrated in a wide range of simulated and real-data examples, and compare favorably to a state of the art flexible regression and classification method in the statistical literature, the Bayesian additive regression trees (BART) method. User-friendly software packages in Matlab, R, and Python implementing the proposed methods are available at https://github.com/VBayesLab.

中文翻译：

---

贝叶斯深网 GLM 和 GLMM

摘要 深度前馈神经网络 (DFNN) 是功能逼近的强大工具。我们描述了包含由 DFNN 形成的基函数的广义线性和广义线性混合模型的灵活版本。考虑具有随机效应的神经网络在文献中并未广泛使用，这可能是因为将特定主题的参数合并到已经很复杂的模型中存在计算挑战。使用高斯变分逼近开发了用于高维贝叶斯推理的高效计算方法，协方差矩阵具有简约但灵活的因子参数化。我们采用自然梯度方法进行优化，利用变分协方差矩阵的因子结构来计算自然梯度。我们灵活的 DFNN 模型和贝叶斯推理方法导致回归和分类方法具有很高的预测精度，并且能够以原则和方便的方式量化预测的不确定性。我们还描述了如何在我们的深度学习方法中执行变量选择。所提出的方法在广泛的模拟和真实数据示例中得到了说明，并且与统计文献中最先进的灵活回归和分类方法贝叶斯加性回归树 (BART) 方法相比具有优势。用户友好的 Matlab、R 和 Python 软件包可在 https://github.com/VBayesLab 上获得。并且能够以一种有原则和方便的方式量化预测的不确定性。我们还描述了如何在我们的深度学习方法中执行变量选择。所提出的方法在广泛的模拟和真实数据示例中得到了说明，并且与统计文献中最先进的灵活回归和分类方法贝叶斯加性回归树 (BART) 方法相比具有优势。用户友好的 Matlab、R 和 Python 软件包可在 https://github.com/VBayesLab 上获得。并且能够以一种有原则和方便的方式量化预测的不确定性。我们还描述了如何在我们的深度学习方法中执行变量选择。所提出的方法在广泛的模拟和真实数据示例中得到了说明，并且与统计文献中最先进的灵活回归和分类方法贝叶斯加性回归树 (BART) 方法相比具有优势。用户友好的 Matlab、R 和 Python 软件包可在 https://github.com/VBayesLab 上获得。并优于统计文献中最先进的灵活回归和分类方法，即贝叶斯加法回归树 (BART) 方法。用户友好的 Matlab、R 和 Python 软件包可在 https://github.com/VBayesLab 上获得。并优于统计文献中最先进的灵活回归和分类方法，即贝叶斯加法回归树 (BART) 方法。用户友好的 Matlab、R 和 Python 软件包可在 https://github.com/VBayesLab 上获得。