Most sequence-to-sequence abstractive summarization models generate the summaries based on the source article and the generated words, but they often neglect the future information implied in the un-generated words, which means that they lack the ability of “lookahead.” In this paper, we present a novel summarization model with “lookahead” ability to fully employ the implied future information. Our model takes two steps: (1) in the first step, an asynchronous decoder model with a no ground truth guiding backward decoder that explicitly produces and exploits the future information is trained. (2) in the inference process, in addition to the joint probability of the generated sequence, an enriched-information decoding method is proposed to further take future ROUGE reward of the un-generated words into account. Furthermore, the future ROUGE reward is predicted by a novel reward-predict model, and it takes the hidden states of the pre-trained asynchronous decoder model as input. Experimental results show that our two-step summarization model achieves new state-of-the-art results on CNN/Daily Mail dataset and the generalization of our model on test-only DUC-2002 datasets achieves higher scores than the state-of-the-art model.

大多数序列到序列的抽象汇总模型都是基于源文章和所生成的单词来生成摘要，但它们通常会忽略未生成的单词中所隐含的未来信息，这意味着它们缺乏“超前”能力。在本文中，我们提出了一种新颖的汇总模型，具有“先行”能力，可以充分利用隐含的未来信息。我们的模型分两个步骤：（1）在第一步中，训练了一个无基础真理的异步解码器模型，该模型指导显式产生和利用未来信息的后向解码器。（2）在推理过程中，除了生成序列的联合概率外，还提出了一种丰富的信息解码方法，以进一步考虑未生成词的未来ROUGE奖励。此外，未来的ROUGE奖励将通过新颖的奖励预测模型进行预测，并将预训练的异步解码器模型的隐藏状态作为输入。实验结果表明，我们的两步汇总模型在CNN / Daily Mail数据集上获得了最新的结果，并且在仅测试的DUC-2002数据集上对该模型的泛化获得了比最新状态更高的分数艺术模型。