Language models are at the heart of numerous works, notably in the text mining and information retrieval communities. These statistical models aim at extracting word distributions, from simple unigram models to recurrent approaches with latent variables that capture subtle dependencies in texts. However, those models are learned from word sequences only, and authors’ identities, as well as publication dates, are seldom considered. We propose a neural model, based on recurrent language modeling (e.g., LSTM), which aims at capturing language diffusion tendencies in author communities through time. By conditioning language models with author and dynamic temporal vector states, we are able to leverage the latent dependencies between the text contexts. The model captures language evolution of authors via a shared temporal prediction function in a latent space, which allows to handle a variety of modeling tasks, including completion and prediction of language models through time. Experiments show the performances of the approach, compared to several temporal and non-temporal language baselines on two real-world corpora.

语言模型是许多作品的核心，尤其是在文本挖掘和信息检索社区。这些统计模型旨在提取单词分布，从简单的字母组合模型到具有潜在变量的递归方法，这些潜在变量可捕获文本中的细微相关性。但是，这些模型仅从单词序列中学习，很少考虑作者的身份以及出版日期。我们提出一种基于递归语言建模（例如LSTM）的神经模型，其目的是捕获随着时间流逝作者社区中的语言传播趋势。通过使用作者和动态时态矢量状态来调节语言模型，我们能够利用文本上下文之间的潜在依赖关系。该模型通过潜在空间中的共享时间预测功能来捕获作者的语言发展，该功能可以处理各种建模任务，包括语言模型的完成和时间预测。与两个真实世界语料库上的几种时态和非时态语言基线相比，实验显示了该方法的性能。