Deep-predictive-coding networks (DPCNs) are hierarchical, generative models that rely on feed-forward and feed-back connections to modulate latent feature representations of stimuli in a dynamic and context-sensitive manner. A crucial element of DPCNs is a forward-backward inference procedure to uncover sparse states of a dynamic model, which are used for invariant feature extraction. However, this inference and the corresponding backwards network parameter updating are major computational bottlenecks. They severely limit the network depths that can be reasonably implemented and easily trained. We therefore propose a optimization strategy, with better empirical and theoretical convergence, based on accelerated proximal gradients. We demonstrate that the ability to construct deeper DPCNs leads to receptive fields that capture well the entire notions of objects on which the networks are trained. This improves the feature representations. It yields completely unsupervised classifiers that surpass convolutional and convolutional-recurrent autoencoders and are on par with convolutional networks trained in a supervised manner. This is despite the DPCNs having orders of magnitude fewer parameters.

中文翻译：

---

深度预测编码网络中的更快收敛，以学习更深入的表示形式

深度预测编码网络（DPCN）是分层的生成模型，其依赖于前馈和反馈连接以动态和上下文相关的方式调制刺激的潜在特征表示。DPCN的关键要素是前向后推过程，以发现动态模型的稀疏状态，该状态用于不变特征提取。但是，此推论和相应的向后网络参数更新是主要的计算瓶颈。它们严重限制了可以合理实施和轻松培训的网络深度。因此，我们基于加速的近端梯度，提出了具有更好的经验和理论收敛性的优化策略。我们证明了构造更深的DPCN的能力导致了可以很好地捕获网络训练对象的整个概念的接受域。这改善了特征表示。它产生了完全不受监督的分类器，这些分类器超过了卷积和卷积递归自动编码器，并且与以监督方式训练的卷积网络相当。尽管DPCN的参数要少几个数量级。