Neural models have become successful at producing abstractive summaries that are human-readable and fluent. However, these models have two critical shortcomings: they often don't respect the facts that are either included in the source article or are known to humans as commonsense knowledge, and they don't produce coherent summaries when the source article is long. In this work, we propose a novel architecture that extends Transformer encoder-decoder architecture in order to improve on these shortcomings. First, we incorporate entity-level knowledge from the Wikidata knowledge graph into the encoder-decoder architecture. Injecting structural world knowledge from Wikidata helps our abstractive summarization model to be more fact-aware. Second, we utilize the ideas used in Transformer-XL language model in our proposed encoder-decoder architecture. This helps our model with producing coherent summaries even when the source article is long. We test our model on CNN/Daily Mail summarization dataset and show improvements on ROUGE scores over the baseline Transformer model. We also include model predictions for which our model accurately conveys the facts, while the baseline Transformer model doesn't.

中文翻译：

---

注意事实：知识提升的连贯抽象文本摘要

神经模型在生成人类可读且流畅的抽象摘要方面取得了成功。然而，这些模型有两个严重的缺点：它们通常不尊重源文章中包含的或作为常识知识为人类所知的事实，并且当源文章很长时，它们不会产生连贯的摘要。在这项工作中，我们提出了一种扩展 Transformer 编码器-解码器架构的新颖架构，以改进这些缺点。首先，我们将来自维基数据知识图的实体级知识合并到编码器-解码器架构中。从维基数据中注入结构世界知识有助于我们的抽象摘要模型更加了解事实。第二，我们在我们提出的编码器-解码器架构中利用了 Transformer-XL 语言模型中使用的思想。即使源文章很长，这也有助于我们的模型生成连贯的摘要。我们在 CNN/Daily Mail 摘要数据集上测试了我们的模型，并展示了 ROUGE 分数相对于基线 Transformer 模型的改进。我们还包括我们的模型准确传达事实的模型预测，而基线 Transformer 模型则没有。