Two popular types of machine translation (MT) are phrase-based and neural machine translation systems. Both of these types of systems are composed of multiple complex models or layers. Each of these models and layers learns different linguistic aspects of the source language. However, for some of these models and layers, it is not clear which linguistic phenomena are learned or how this information is learned. For phrase-based MT systems, it is often clear what information is learned by each model, and the question is rather how this information is learned, especially for its phrase reordering model. For neural machine translation systems, the situation is even more complex, since for many cases it is not exactly clear what information is learned and how it is learned. To shed light on what linguistic phenomena are captured by MT systems, we analyze the behavior of important models in both phrase-based and neural MT systems. We consider phrase reordering models from phrase-based MT systems to investigate which words from inside of a phrase have the biggest impact on defining the phrase reordering behavior. Additionally, to contribute to the interpretability of neural MT systems we study the behavior of the attention model, which is a key component in neural MT systems and the closest model in functionality to phrase reordering models in phrase-based systems. The attention model together with the encoder hidden state representations form the main components to encode source side linguistic information in neural MT. To this end, we also analyze the information captured in the encoder hidden state representations of a neural MT system. We investigate the extent to which syntactic and lexical-semantic information from the source side is captured by hidden state representations of different neural MT architectures.

中文翻译：

---

基于短语和神经机器翻译的实证分析

机器翻译（MT）的两种流行类型是基于短语的翻译系统和神经机器翻译系统。这两种类型的系统都由多个复杂的模型或层组成。这些模型和层中的每一个都学习源语言的不同语言方面。但是，对于其中一些模型和层，尚不清楚学习哪些语言现象或如何学习此信息。对于基于短语的MT系统，通常很清楚每种模型都学习了哪些信息，而问题在于，如何学习此信息，尤其是对于短语重新排序模型而言。对于神经机器翻译系统，情况甚至更为复杂，因为在许多情况下，尚不清楚确切地了解到什么信息以及如何学习。为了阐明MT系统捕获的语言现象，我们分析了基于短语的MT系统和神经MT系统中重要模型的行为。我们考虑基于短语的MT系统中的短语重排序模型，以研究短语内部的哪些单词对定义短语重排序行为的影响最大。此外，为了提高神经MT系统的可解释性，我们研究了注意力模型的行为，该模型是神经MT系统的关键组成部分，并且是与基于短语的系统中的短语重排模型最接近的功能模型。注意模型与编码器隐藏状态表示一起构成对神经MT中的源侧语言信息进行编码的主要组件。为此，我们还将分析在神经MT系统的编码器隐藏状态表示中捕获的信息。