Abstract Sequence labeling is a fundamental task in natural language processing and has been widely studied. Recently, RNN-based sequence labeling models have increasingly gained attentions. Despite superior performance achieved by learning the long short-term (i.e., successive) dependencies, the way of sequentially processing inputs might limit the ability to capture the non-continuous relations over tokens within a sentence. To tackle the problem, we focus on how to effectively model successive and discrete dependencies of each token for enhancing the sequence labeling performance. Specifically, we propose an innovative attention-based model (called position-aware self-attention , i.e., PSA ) as well as a well-designed self-attentional context fusion layer within a neural network architecture, to explore the positional information of an input sequence for capturing the latent relations among tokens. Extensive experiments on three classical tasks in sequence labeling domain, i.e., part-of-speech (POS) tagging, named entity recognition (NER) and phrase chunking, demonstrate our proposed model outperforms the state-of-the-arts without any external knowledge, in terms of various metrics.

中文翻译：

---

基于位置感知自注意力的神经序列标记

摘要 序列标注是自然语言处理中的一项基本任务，已被广泛研究。最近，基于 RNN 的序列标记模型越来越受到关注。尽管通过学习长期短期（即连续）依赖关系获得了卓越的性能，但顺序处理输入的方式可能会限制捕获句子中标记上的非连续关系的能力。为了解决这个问题，我们专注于如何有效地对每个标记的连续和离散依赖项进行建模，以提高序列标记性能。具体来说，我们提出了一种创新的基于注意力的模型（称为位置感知自注意力，即 PSA）以及神经网络架构中精心设计的自注意力上下文融合层，探索输入序列的位置信息以捕获标记之间的潜在关系。在序列标记领域的三个经典任务上的大量实验，即词性 (POS) 标记、命名实体识别 (NER) 和短语组块，证明我们提出的模型在没有任何外部知识的情况下优于最先进的模型，就各种指标而言。