Deep unfolding provides an approach to integrate the probabilistic generative models and the deterministic neural networks. Such an approach is benefited by deep representation, easy interpretation, flexible learning and stochastic modeling. This study develops the unsupervised and supervised learning of deep unfolded topic models for document representation and classification. Conventionally, the unsupervised and supervised topic models are inferred via the variational inference algorithm where the model parameters are estimated by maximizing the lower bound of logarithm of marginal likelihood using input documents without and with class labels, respectively. The representation capability or classification accuracy is constrained by the variational lower bound and the tied model parameters across inference procedure. This paper aims to relax these constraints by directly maximizing the end performance criterion and continuously untying the parameters in learning process via deep unfolding inference (DUI). The inference procedure is treated as the layer-wise learning in a deep neural network. The end performance is iteratively improved by using the estimated topic parameters according to the exponentiated updates. Deep learning of topic models is therefore implemented through a back-propagation procedure. Experimental results show the merits of DUI with increasing number of layers compared with variational inference in unsupervised as well as supervised topic models.

中文翻译：

---

主题模型的深入展示

深度展开提供了一种整合概率生成模型和确定性神经网络的方法。这种方法得益于深入的表示，易于解释的方法，灵活的学习方法和随机建模方法。这项研究开发了用于文档表示和分类的深层展开主题模型的无监督学习。传统上，无监督和受监督主题模型是通过变分推理算法来推断的，其中模型参数是通过分别使用不带类别标签和带类别标签的输入文档通过最大化边际似然对数的下界来估计的。表示能力或分类精度受变异过程的下界和整个推理过程中受束缚的模型参数的约束。本文旨在通过直接最大化最终性能标准并通过深度展开推理（DUI）在学习过程中连续解开参数来放松这些约束。推理过程被视为深度神经网络中的分层学习。通过根据指数更新使用估计的主题参数，迭代地提高了最终性能。因此，主题模型的深度学习是通过反向传播过程实现的。实验结果表明，在无监督和有监督的主题模型中，与变分推理相比，DUI的优点在于层数增加。推理过程被视为深度神经网络中的分层学习。通过根据指数更新使用估计的主题参数，迭代地提高了最终性能。因此，主题模型的深度学习是通过反向传播过程实现的。实验结果表明，在无监督和有监督的主题模型中，与变分推理相比，DUI的优点在于层数增加。推理过程被视为深度神经网络中的分层学习。通过根据指数更新使用估计的主题参数，迭代地提高了最终性能。因此，主题模型的深度学习是通过反向传播过程实现的。实验结果表明，在无监督和有监督的主题模型中，与变分推理相比，DUI的优点在于层数增加。