当前位置: X-MOL 学术Int. J. Artif. Intell. Tools › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Language Model Pre-training Method in Machine Translation Based on Named Entity Recognition
International Journal on Artificial Intelligence Tools ( IF 1.1 ) Pub Date : 2020-11-30 , DOI: 10.1142/s0218213020400217
Zhen Li 1 , Dan Qu 1 , Chaojie Xie 2 , Wenlin Zhang 1 , Yanxia Li 3
Affiliation  

Neural Machine Translation (NMT) model has become the mainstream technology in machine translation. The supervised neural machine translation model trains with abundant of sentence-level parallel corpora. But for low-resources language or dialect with no such corpus available, it is difficult to achieve good performance. Researchers began to focus on unsupervised neural machine translation (UNMT) that monolingual corpus as training data. UNMT need to construct the language model (LM) which learns semantic information from the monolingual corpus. This paper focuses on the pre-training of LM in unsupervised machine translation and proposes a pre-training method, NER-MLM (named entity recognition masked language model). Through performing NER, the proposed method can obtain better semantic information and language model parameters with better training results. In the unsupervised machine translation task, the BLEU scores on the WMT’16 English–French, English–German, data sets are 35.30, 27.30 respectively. To the best of our knowledge, this is the highest results in the field of UNMT reported so far.

中文翻译:

基于命名实体识别的机器翻译语言模型预训练方法

神经机器翻译(NMT)模型已成为机器翻译的主流技术。监督神经机器翻译模型训练有丰富的句子级并行语料库。但是对于没有这样的语料库的资源匮乏的语言或方言,很难达到良好的性能。研究人员开始关注以单语语料库作为训练数据的无监督神经机器翻译(UNMT)。UNMT 需要构建从单语语料库中学习语义信息的语言模型(LM)。本文重点关注LM在无监督机器翻译中的预训练,提出了一种预训练方法NER-MLM(命名实体识别掩码语言模型)。通过执行NER,该方法可以获得更好的语义信息和语言模型参数,训练效果更好。在无监督机器翻译任务中,WMT'16 英法、英德数据集上的 BLEU 分数分别为 35.30、27.30。据我们所知,这是迄今为止在 UNMT 领域报告的最高结果。
更新日期:2020-11-30
down
wechat
bug