Texts are the major information carrier for internet users, from which learning the latent representations has important research and practical value. Neural topic models have been proposed and have great performance in extracting interpretable latent topics and representations of texts. However, there remain two major limitations: (1) these methods generally ignore the contextual information of texts and have limited feature representation ability due to the shallow feed-forward network architecture, (2) Sparsity of the representations in topic semantic space is ignored. To address these issues, in this paper, we propose a semantic reinforcement neural variational sparse topic model (SR-NSTM) towards explainable and sparse latent text representation learning. Compared with existing neural topic models, SR-NSTM models the generative process of texts with probabilistic distributions parameterized with neural networks and incorporates Bi-directional LSTM to embed contextual information at the document level. It achieves sparse posterior representations over documents and words with zero-mean Laplace distribution and topics with sparsemax. Moreover, we propose a supervised extension of SR-NSTM via adding the max-margin posterior regularization to tackle the supervised tasks. The neural variational inference method is utilized to learn our models efficiently. Experimental results on Web Snippets, 20Newsgroups, BBC, and Biomedical datasets demonstrate that the contextual information and revisiting generative process can improve the performance, leading to the competitive performance of our models in learning coherent topics and explainable sparse representations for texts.

文本是互联网用户的主要信息载体，从中学习潜在的表征具有重要的研究意义和实用价值。已经提出了神经主题模型，并且该神经模型在提取可解释的潜在主题和文本表示方面具有出色的性能。但是，仍然存在两个主要局限性：（1）由于浅层前馈网络体系结构，这些方法通常忽略文本的上下文信息并且具有有限的特征表示能力，（2）主题语义空间中表示的稀疏性被忽略。为了解决这些问题，在本文中，我们针对可解释和稀疏的潜在文本表示学习提出了一种语义增强神经变分稀疏主题模型（SR-NSTM）。与现有的神经主题模型相比，SR-NSTM使用神经网络对具有概率分布的文本的生成过程进行建模，并结合了双向LSTM在文档级别嵌入上下文信息。它实现了零均值拉普拉斯分布的文档和单词的稀疏后验表示以及具有sparsemax的主题的稀疏后验表示。此外，我们通过增加max-margin后验正则化来提出SR-NSTM的监督扩展，以解决监督任务。利用神经变分推理方法来有效地学习我们的模型。网络摘要，20Newsgroups，BBC和Biomedical数据集上的实验结果表明，上下文信息和重新产生过程可以提高性能，