Named entity recognition (NER) is the basis for many natural language processing (NLP) tasks such as information extraction and question answering. The accuracy of the NER directly affects the results of downstream tasks. Most of the relevant methods are implemented using neural networks, however, the word vectors obtained from a small data set cannot describe unusual, previously-unseen entities accurately and the results are not sufficiently accurate. Recently, the use of XLNet as a new pre-trained model has yielded satisfactory results in many NLP tasks, integration of XLNet embeddings in existent NLP tasks is not straightforward. In this paper, a new neural network model is proposed to improve the effectiveness of the NER by using a pre-trained XLNet, bi-directional long-short term memory (Bi-LSTM) and conditional random field (CRF). Pre-trained XLNet model is used to extract sentence features, then the classic NER neural network model is combined with the obtained features. In addition, the superiority of XLNet in NER tasks is demonstrated. We evaluate our model on the CoNLL-2003 English dataset and WNUT-2017 and show that the XLNet-BiLSTM-CRF obtains state-of-the-art results.

命名实体识别 (NER) 是许多自然语言处理 (NLP) 任务（例如信息提取和问答）的基础。NER的准确性直接影响下游任务的结果。大多数相关方法都是使用神经网络实现的，但是，从小数据集获得的词向量无法准确描述异常的、以前未见过的实体，结果不够准确。最近，使用 XLNet 作为新的预训练模型在许多 NLP 任务中取得了令人满意的结果，在现有 NLP 任务中集成 XLNet 嵌入并不简单。在本文中，提出了一种新的神经网络模型，通过使用预训练的 XLNet、双向长短期记忆 (Bi-LSTM) 和条件随机场 (CRF) 来提高 NER 的有效性。使用预训练的 XLNet 模型提取句子特征，然后将经典的 NER 神经网络模型与获得的特征相结合。此外，证明了 XLNet 在 NER 任务中的优越性。我们在 CoNLL-2003 英语数据集和 WNUT-2017 上评估我们的模型，并表明 XLNet-BiLSTM-CRF 获得了最先进的结果。