Generative commonsense reasoning which aims to empower machines to generate sentences with the capacity of reasoning over a set of concepts is a critical bottleneck for text generation. Even the state-of-the-art pre-trained language generation models struggle at this task and often produce implausible and anomalous sentences. One reason is that they rarely consider incorporating the knowledge graph which can provide rich relational information among the commonsense concepts. To promote the ability of commonsense reasoning for text generation, we propose a novel knowledge graphaugmented pre-trained language generation model KG-BART, which encompasses the complex relations of concepts through the knowledge graph and produces more logical and natural sentences as output. Moreover, KG-BART can leverage the graph attention to aggregate the rich concept semantics that enhances the model generalization on unseen concept sets. Experiments on benchmark CommonGen dataset verify the effectiveness of our proposed approach by comparing with several strong pre-trained language generation models, particularly KG-BART outperforms BART by 15.98%, 17.49%, in terms of BLEU-3, 4. Moreover, we also show that the generated context by our model can work as background scenarios to benefit downstream commonsense QA tasks.

中文翻译：

---

KG-BART：用于生成常识推理的知识图增强 BART

生成常识推理旨在使机器能够生成具有推理一组概念的能力的句子，这是文本生成的关键瓶颈。即使是最先进的预训练语言生成模型也很难完成这项任务，并且经常会产生令人难以置信和异常的句子。原因之一是他们很少考虑合并可以在常识概念之间提供丰富关系信息的知识图谱。为了提高文本生成的常识推理能力，我们提出了一种新颖的知识图增强预训练语言生成模型 KG-BART，它通过知识图包含概念的复杂关系，并产生更多逻辑和自然的句子作为输出。而且，KG-BART 可以利用图注意力来聚合丰富的概念语义，从而增强对未见概念集的模型泛化。在基准 CommonGen 数据集上的实验通过与几个强大的预训练语言生成模型进行比较来验证我们提出的方法的有效性，特别是 KG-BART 在 BLEU-3、4 方面的性能比 BART 高 15.98%、17.49%。此外，我们还表明我们的模型生成的上下文可以作为背景场景，使下游常识 QA 任务受益。