Unsupervised neural machine translation (UNMT) has achieved remarkable results for several language pairs, such as French–English and German–English. Most previous studies have focused on modeling UNMT systems; few studies have investigated the effect of UNMT on specific languages. In this article, we first empirically investigate UNMT for four diverse language pairs (French/German/Chinese/Japanese–English). We confirm that the performance of UNMT in translation tasks for similar language pairs (French/German–English) is dramatically better than for distant language pairs (Chinese/Japanese–English). We empirically show that the lack of shared words and different word orderings are the main reasons that lead UNMT to underperform in Chinese/Japanese–English. Based on these findings, we propose several methods, including artificial shared words and pre-ordering, to improve the performance of UNMT for distant language pairs. Moreover, we propose a simple general method to improve translation performance for all these four language pairs. The existing UNMT model can generate a translation of a reasonable quality after a few training epochs owing to a denoising mechanism and shared latent representations. However, learning shared latent representations restricts the performance of translation in both directions, particularly for distant language pairs, while denoising dramatically delays convergence by continuously modifying the training data. To avoid these problems, we propose a simple, yet effective and efficient, approach that (like UNMT) relies solely on monolingual corpora: pseudo-data-based unsupervised neural machine translation. Experimental results for these four language pairs show that our proposed methods significantly outperform UNMT baselines.

中文翻译：

---

相似和遥远语言对的无监督神经机器翻译

无监督神经机器翻译 (UNMT) 在多种语言对（例如法语-英语和德语-英语）上取得了显著成果。以前的大多数研究都集中在建模 UNMT 系统上。很少有研究调查 UNMT 对特定语言的影响。在本文中，我们首先对四种不同语言对（法语/德语/汉语/日语-英语）的 UNMT 进行实证研究。我们确认，UNMT 在相似语言对（法语/德语-英语）的翻译任务中的性能明显优于远距离语言对（汉语/日语-英语）。我们凭经验表明，缺乏共享词和不同的词序是导致 UNMT 在汉语/日语-英语中表现不佳的主要原因。基于这些发现，我们提出了几种方法，包括人工共享词和预排序，以提高 UNMT 对远程语言对的性能。此外，我们提出了一种简单的通用方法来提高所有这四种语言对的翻译性能。由于去噪机制和共享的潜在表示，现有的 UNMT 模型可以在几个训练时期后生成合理质量的翻译。然而，学习共享潜在表示限制了双向翻译的性能，特别是对于远程语言对，而去噪通过不断修改训练数据显着延迟了收敛。为了避免这些问题，我们提出了一种简单但有效且高效的方法，该方法（如 UNMT）仅依赖于单语语料库：基于伪数据的无监督神经机器翻译。