当前位置: X-MOL 学术arXiv.cs.CL › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Exploring and Improving Robustness of Multi Task Deep Neural Networks via Domain Agnostic Defenses
arXiv - CS - Computation and Language Pub Date : 2020-01-11 , DOI: arxiv-2001.05286
Kashyap Coimbatore Murali

In this paper, we explore the robustness of the Multi-Task Deep Neural Networks (MT-DNN) against non-targeted adversarial attacks across Natural Language Understanding (NLU) tasks as well as some possible ways to defend against them. Liu et al., have shown that the Multi-Task Deep Neural Network, due to the regularization effect produced when training as a result of its cross task data, is more robust than a vanilla BERT model trained only on one task (1.1%-1.5% absolute difference). We further show that although the MT-DNN has generalized better, making it easily transferable across domains and tasks, it can still be compromised as after only 2 attacks (1-character and 2-character) the accuracy drops by 42.05% and 32.24% for the SNLI and SciTail tasks. Finally, we propose a domain agnostic defense which restores the model's accuracy (36.75% and 25.94% respectively) as opposed to a general-purpose defense or an off-the-shelf spell checker.

中文翻译:

通过领域不可知防御探索和提高多任务深度神经网络的鲁棒性

在本文中,我们探讨了多任务深度神经网络 (MT-DNN) 针对跨自然语言理解 (NLU) 任务的非目标对抗性攻击的鲁棒性,以及一些可能的防御方法。Liu 等人的研究表明,多任务深度神经网络由于其跨任务数据在训练时产生的正则化效应,比仅在一项任务上训练的普通 BERT 模型更稳健(1.1%- 1.5% 的绝对差异)。我们进一步表明,尽管 MT-DNN 具有更好的泛化能力,使其易于跨领域和任务转移,但它仍然可能受到损害,因为仅在 2 次攻击(1 个字符和 2 个字符)后,准确度下降了 42.05% 和 32.24%用于 SNLI 和 SciTail 任务。最后,我们提出了一种域不可知防御,可以恢复模型的准确性 (36.
更新日期:2020-01-16
down
wechat
bug