Transformers are arguably the main workhorse in recent Natural Language Processing research. By definition a Transformer is invariant with respect to reorderings of the input. However, language is inherently sequential and word order is essential to the semantics and syntax of an utterance. In this paper, we provide an overview of common methods to incorporate position information into Transformer models. The objectives of this survey are to i) showcase that position information in Transformer is a vibrant and extensive research area; ii) enable the reader to compare existing methods by providing a unified notation and meaningful clustering; iii) indicate what characteristics of an application should be taken into account when selecting a position encoding; iv) provide stimuli for future research.

中文翻译：

---

变压器中的位置信息：概述

变压器可以说是最近自然语言处理研究的主要动力。根据定义，变压器对于输入的重新排序是不变的。但是，语言本质上是顺序的，并且单词顺序对于发声的语义和语法至关重要。在本文中，我们概述了将位置信息合并到Transformer模型中的常用方法。这项调查的目的是：i）展示《变形金刚》中的位置信息是一个充满活力且广泛的研究领域；ii）通过提供统一的符号和有意义的聚类，使读者能够比较现有方法；iii）指出在选择位置编码时应考虑到应用程序的哪些特性；iv）为将来的研究提供刺激。