当前位置: X-MOL 学术arXiv.cs.AI › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Minimize Exposure Bias of Seq2Seq Models in Joint Entity and Relation Extraction
arXiv - CS - Artificial Intelligence Pub Date : 2020-09-16 , DOI: arxiv-2009.07503
Ranran Haoran Zhang, Qianying Liu, Aysa Xuemo Fan, Heng Ji, Daojian Zeng, Fei Cheng, Daisuke Kawahara and Sadao Kurohashi

Joint entity and relation extraction aims to extract relation triplets from plain text directly. Prior work leverages Sequence-to-Sequence (Seq2Seq) models for triplet sequence generation. However, Seq2Seq enforces an unnecessary order on the unordered triplets and involves a large decoding length associated with error accumulation. These introduce exposure bias, which may cause the models overfit to the frequent label combination, thus deteriorating the generalization. We propose a novel Sequence-to-Unordered-Multi-Tree (Seq2UMTree) model to minimize the effects of exposure bias by limiting the decoding length to three within a triplet and removing the order among triplets. We evaluate our model on two datasets, DuIE and NYT, and systematically study how exposure bias alters the performance of Seq2Seq models. Experiments show that the state-of-the-art Seq2Seq model overfits to both datasets while Seq2UMTree shows significantly better generalization. Our code is available at https://github.com/WindChimeRan/OpenJERE .

中文翻译:

最小化联合实体和关系提取中 Seq2Seq 模型的暴露偏差

联合实体和关系提取旨在直接从纯文本中提取关系三元组。先前的工作利用序列到序列 (Seq2Seq) 模型来生成三元组序列。然而,Seq2Seq 对无序三元组强制执行不必要的顺序,并且涉及与错误累积相关的大解码长度。这些引入了曝光偏差,这可能会导致模型过度拟合频繁的标签组合,从而降低泛化能力。我们提出了一种新的序列到无序多树 (Seq2UMTree) 模型,通过将三元组中的解码长度限制为三个并去除三元组之间的顺序来最小化曝光偏差的影响。我们在两个数据集 DuIE 和 NYT 上评估我们的模型,并系统地研究曝光偏差如何改变 Seq2Seq 模型的性能。实验表明,最先进的 Seq2Seq 模型对两个数据集都过拟合,而 Seq2UMTree 显示出明显更好的泛化。我们的代码可在 https://github.com/WindChimeRan/OpenJERE 获得。
更新日期:2020-10-07
down
wechat
bug