At present, the commonly used word vector methods for news text classification mostly adopt Word2Vec, Glove, Bert and other word vector models, ignoring the remote context connections of Chinese text itself. TextCNN, RNN, BiLSTM and other neural network classification models lack the extraction of important information features of the text, and the word dependence inside the text is not strong, resulting in inaccurate classification results. To solve the above problems, this paper proposes the DPCNN-Attention news text classification model based on ERNIE’s pre-training model(EDA for short, the following general). The DPCNN neural network model with Mish() activation function is adopted to obtain the maximum length of semantic association between long distance texts in news texts. By adding attention mechanism into EDA model, in the feature extraction process, according to the importance of words to the classification results, different weights are assigned to them to enhance the word dependence relationship within the text, thus greatly improving the classification accuracy. The EDA model was experimentally verified on the THUCNews dataset. The results showed that the EDA model improved by about 6% compared with BERT’s pre-training model, and 0.4% compared with ERNIE’s pre-training model. The loss rate decreased by 0.2 compared with BERT and 0.01 compared with ERNIE’s.

目前，新闻文本分类常用的词向量方法大多采用Word2Vec、Glove、Bert等词向量模型，忽略了中文文本本身的远程上下文联系。TextCNN、RNN、BiLSTM等神经网络分类模型缺乏对文本重要信息特征的提取，文本内部的单词依赖性不强，导致分类结果不准确。针对上述问题，本文提出了基于ERNIE的预训练模型（简称EDA，以下通用）的DPCNN-Attention新闻文本分类模型。采用具有 Mish() 激活函数的 DPCNN 神经网络模型来获得新闻文本中长距离文本之间语义关联的最大长度。通过在 EDA 模型中加入注意力机制，在特征提取过程中，根据词对分类结果的重要性，赋予它们不同的权重，增强文本内的词依赖关系，从而大大提高分类准确率。EDA 模型在 THUCNews 数据集上进行了实验验证。结果表明，EDA模型与BERT的预训练模型相比提升了约6%，与ERNIE的预训练模型相比提升了0.4%。丢失率与 BERT 相比下降了 0.2，与 ERNIE 相比下降了 0.01。结果表明，EDA模型与BERT的预训练模型相比提升了约6%，与ERNIE的预训练模型相比提升了0.4%。丢失率与 BERT 相比下降了 0.2，与 ERNIE 相比下降了 0.01。结果表明，EDA模型与BERT的预训练模型相比提升了约6%，与ERNIE的预训练模型相比提升了0.4%。丢失率与 BERT 相比下降了 0.2，与 ERNIE 相比下降了 0.01。