当前位置: X-MOL 学术arXiv.cs.CL › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Synthesizing Adversarial Negative Responses for Robust Response Ranking and Evaluation
arXiv - CS - Computation and Language Pub Date : 2021-06-10 , DOI: arxiv-2106.05894
Prakhar Gupta, Yulia Tsvetkov, Jeffrey P. Bigham

Open-domain neural dialogue models have achieved high performance in response ranking and evaluation tasks. These tasks are formulated as a binary classification of responses given in a dialogue context, and models generally learn to make predictions based on context-response content similarity. However, over-reliance on content similarity makes the models less sensitive to the presence of inconsistencies, incorrect time expressions and other factors important for response appropriateness and coherence. We propose approaches for automatically creating adversarial negative training data to help ranking and evaluation models learn features beyond content similarity. We propose mask-and-fill and keyword-guided approaches that generate negative examples for training more robust dialogue systems. These generated adversarial responses have high content similarity with the contexts but are either incoherent, inappropriate or not fluent. Our approaches are fully data-driven and can be easily incorporated in existing models and datasets. Experiments on classification, ranking and evaluation tasks across multiple datasets demonstrate that our approaches outperform strong baselines in providing informative negative examples for training dialogue systems.

中文翻译:

综合对抗性负面反应以进行稳健的反应排名和评估

开放域神经对话模型在响应排序和评估任务中取得了高性能。这些任务被表述为对话上下文中给出的响应的二元分类,模型通常学习基于上下文响应内容的相似性进行预测。然而,过度依赖内容相似性会使模型对不一致、不正确的时间表达和其他对响应适当性和连贯性很重要的因素的存在不那么敏感。我们提出了自动创建对抗性负面训练数据的方法,以帮助排名和评估模型学习内容相似性之外的特征。我们提出了掩码填充和关键字引导的方法,这些方法可以生成用于训练更强大的对话系统的负面示例。这些生成的对抗性响应与上下文具有很高的内容相似性,但要么不连贯、不恰当或不流畅。我们的方法完全是数据驱动的,可以很容易地合并到现有的模型和数据集中。跨多个数据集的分类、排名和评估任务的实验表明,我们的方法在为训练对话系统提供信息丰富的负面示例方面优于强大的基线。
更新日期:2021-06-11
down
wechat
bug