Global vectors, or global embeddings, are important word representations for many natural language processing tasks. With the popularity of dynamic embeddings (also known as contextual embeddings, such as ELMo and BERT) in recent years, attentions on global vectors have been diverted to a large extent. While, compared to the dynamic embeddings, the global embeddings are faster to train, straightforward to interpret, and eligible to be evaluated by many standard and credible intrinsic benchmarks (e.g., word similarity correlation and analogy accuracy). Thus, they are still widely-used in numerous downstream applications until now. However, the model design of the global embeddings has some limitations, making the learned word representations suboptimal. In this paper, we propose a novel method to deal with these limitations using PID control. To the best of our knowledge, this is one of the first efforts to leverage PID control in the research of word embeddings. Empirical results on standard intrinsic and extrinsic benchmarks show consistent performance boost of the proposed method, suggesting that the method proposed in this paper can be considered as a promising alternative to learn better word representations for the downstream tasks.

全局向量或全局嵌入是许多自然语言处理任务的重要单词表示形式。近年来，随着动态嵌入（也称为上下文嵌入，例如ELMo和BERT）的流行，对全局向量的关注已在很大程度上转移了。与动态嵌入相比，全局嵌入的训练速度更快，易于解释，并且可以通过许多标准且可信的内在基准进行评估（例如，单词相似性相关性和类比准确性）。因此，到目前为止，它们仍在众多下游应用中得到广泛使用。但是，全局嵌入的模型设计有一些局限性，使学习的单词表示不理想。在本文中，我们提出了一种使用PID控制来解决这些限制的新颖方法。据我们所知，这是在词嵌入研究中利用PID控制的首批尝试之一。标准内在和外在基准的经验结果表明，该方法具有一致的性能提升，这表明本文中提出的方法可以被认为是一种有前途的替代方法，可以为下游任务学习更好的单词表示形式。