In this paper, we propose a method of learning representation layers with squashing activation functions within a deep artificial neural network which directly addresses the vanishing gradients problem. The proposed solution is derived from solving the maximum likelihood estimator for components of the posterior representation, which are approximately Beta-distributed, formulated in the context of variational inference. This approach not only improves the performance of deep neural networks with squashing activation functions on some of the hidden layers - including in discriminative learning - but can be employed towards producing sparse codes.

中文翻译：

---

用人工神经网络学习平面表示

在本文中，我们提出了一种在深层人工神经网络中学习具有挤压激活函数的表示层的方法，该方法直接解决了消失梯度问题。所提出的解决方案是通过求解后验表示的分量的最大似然估计量而得出的，这些分量近似为Beta分布，是在变分推理的情况下制定的。这种方法不仅提高了深层神经网络的性能，而且在某些隐藏层上（包括在判别式学习中）具有挤压激活功能，而且可以用于生成稀疏代码。