Story generation, namely generating a reasonable story from a leading context, is an important but challenging task. In spite of the success in modeling fluency and local coherence, existing neural language generation models (e.g., GPT-2) still suffer from repetition, logic conflicts, and lack of long-range coherence in generated stories. We conjecture that this is because of the difficulty of associating relevant commonsense knowledge, understanding the causal relationships, and planning entities and events with proper temporal order. In this paper, we devise a knowledge-enhanced pretraining model for commonsense story generation. We propose to utilize commonsense knowledge from external knowledge bases to generate reasonable stories. To further capture the causal and temporal dependencies between the sentences in a reasonable story, we employ multi-task learning which combines a discriminative objective to distinguish true and fake stories during fine-tuning. Automatic and manual evaluation shows that our model can generate more reasonable stories than state-of-the-art baselines, particularly in terms of logic and global coherence.

中文翻译：

---

常识故事生成的知识增强预训练模型

故事生成，即从领先的上下文中生成合理的故事，是一项重要但具有挑战性的任务。尽管在建模流畅性和局部连贯性方面取得了成功，但现有的神经语言生成模型（例如 GPT-2）仍然存在重复、逻辑冲突以及生成的故事缺乏长期连贯性的问题。我们推测这是因为关联相关常识知识、理解因果关系以及以适当的时间顺序规划实体和事件的难度。在本文中，我们为常识故事生成设计了一个知识增强的预训练模型。我们建议利用来自外部知识库的常识知识来生成合理的故事。为了进一步捕捉合理故事中句子之间的因果关系和时间依赖性，我们采用多任务学习，结合判别目标在微调期间区分真假故事。自动和手动评估表明，我们的模型可以生成比最先进的基线更合理的故事，尤其是在逻辑和全局一致性方面。