Skip to main content
SpringerLink
Log in

Chinese text classification based on attention mechanism and feature-enhanced fusion neural network

  • Published:
Computing Aims and scope Submit manuscript

Abstract

Owing to the uneven distribution of key features in Chinese texts, key features play different roles in text recognition in Chinese text classification tasks. We propose a feature-enhanced fusion model based on attention mechanism for Chinese text classification, a long short-term memory (LSTM) network, a convolutional neural network (CNN), and a feature-difference enhancement attention algorithm model. The Chinese text is digitized into a vector form containing certain semantic context information into the embedding layer to train and test the neural network by preprocessing. The feature-enhanced fusion model is implemented by double-layer LSTM and CNN modules to enhance the fusion of text features extracted from the attention mechanism for classifying the classifiers. The feature-difference enhancement attention algorithm model not only adds more weight to important text features but also strengthens the differences between them and other text features. This operation can further improves the effect of important features on Chinese text recognition. The two models are classified by the softmax function. The text classification experiments are conducted based on the Chinese text corpus. The experimental results show that compared with the contrast model, the proposed algorithm can significantly improve the recognition ability of Chinese text features.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Chen W, Xu B (2015) Semi-supervised Chinese word segmentation based on bilingual information. In: Proceedings of the 2015 conference on empirical methods in natural language processing, Lisbon, pp 1207–1216

  2. Chen XC, Qiu XP, Zhu CX et al (2015) Long short-term memory neural networks for Chinese word segmentation. In; Proceedings of the 2015 conference on empirical methods in natural language processing, Lisbon, pp 1197–1206

  3. Golub D, He X (2016) Character-level question answering with attention. In: Proceedings of the 2016 conference on empirical methods in natural language processing, Austin, pp 1598–1607

  4. Zhou X, Wan X, Xiao J (2016) Attention-based LSTM network for cross-lingual sentiment classification. In: Proceedings of the 2016 conference on empirical methods in natural language processing, Austin, pp 247–256

  5. Dhingra B, Liu H, Yang Z et al (2017) Gated-attention readers for text comprehension. In: Proceedings of the 55th annual meeting of the association for computational linguistics, Vancouver, pp 1832–1846

  6. Zhang Z, Liu S, Li M et al (2017) Stack-based multi-layer attention for transition-based dependency parsing. In: Proceedings of the 2017 conference on empirical methods in natural language processing, Copenhagen, pp 1677–1682

  7. Kim Y (2014) Convolutional neural networks for sentence classification. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), Doha, pp 1746–1751

  8. Pham N-Q, Kruszewski G, Boleda G (2016) Convolutional neural network language models. In: Proceedings of the 2016 conference on empirical methods in natural language processing, Austin, pp 1153–1162

  9. He H, Gimpel K, Lin J (2015) Multi-perspective sentence similarity modeling with convolutional neural networks. In: Proceedings of the 2015 conference on empirical methods in natural language processing, Lisbon, pp 1576–1586

  10. Lei T, Barzilay R, Jaakkola T (2015) Molding CNNs for text: non-linear, non-consecutive convolutions. In: Proceedings of the 2015 conference on empirical methods in natural language processing, Lisbon, pp 1565–1575

  11. Chen Z, Xu R, Gui L et al (2015) Combining convolutional neural networks and word sentiment sequence features for Chinese text sentiment analysis. J Chin Inf Process 29(6):172–178

    Google Scholar 

  12. Li S, Zhao Z, Liu T et al (2017) Initializing convolutional filters with semantic features for text classification. In: Proceedings of the 2017 conference on empirical methods in natural language processing, Copenhagen, pp 1884–1889

  13. Zhang Y, Marshall I, Wallace BC (2016) Rational-augmented convolutional neural networks for text classification. In: Proceedings of the 2016 conference on empirical methods in natural language processing, Austin, pp 759–804

  14. Miyamoto Y, Cho K (2016) Gated word-character recurrent language model. In: Proceedings of the 2016 conference on empirical methods in natural language processing, Austin, pp 1992–1997

  15. Tang D, Qin B, Liu T et al (2015) Document modeling with gated recurrent neural network for sentiment classification. In: Proceedings of the 2015 conference on empirical methods in natural language processing, Austin, pp 1422–1432

  16. Xu J, Chen D, Qiu X et al (2016) Cached long short-term memory neural networks for document-level sentiment classification. In: Proceedings of the 2016 conference on empirical methods in natural language processing, Austin, pp 1660–1669

  17. Jagannatha AN, Yu H (2016) Structured prediction models for RNN based sequence labeling in clinical text. In: Proceedings of the 2016 conference on empirical methods in natural language processing, Austin, pp 856–865

  18. Liu P, Qiu X, Zhou Y et al (2016) Modelling interaction of sentence pair with coupled-LSTMs. In: Proceedings of the 2016 conference on empirical methods in natural language processing, Austin, pp 1703–1712

  19. An B, Han X, Sun L et al (2016) Triple classification based on synthesized features for knowledge base. J Chin Inf Process 30(6):84–89

    Google Scholar 

  20. Ma J, Pei J, Huang D (2016) Identification of English functional noun phrases by CRFs and the semantic information. J Chin Inf Process 30(6):59–66

    Google Scholar 

  21. Gao C, Xu H, Gao K (2018) Chinese text classification of bidirectional LSTM based on attention mechanism based on part of speech information. J Hebei Univ Sci Technol 39(5):447–454

    Google Scholar 

  22. Kobayashi VB, Mol ST, Berkers HA et al (2018) Text classification for organizational researchers. Organ Res Methods 21(3):766–799

    Article  Google Scholar 

Download references

Funding

This research was supported by the National Natural Science Foundation of China (Grant No. 51805120), Natural Science Foundation of Heilongjiang Province (LH2019E058), University Nursing Program for Young Scholars with Creative Talents in Heilongjiang Province (Grant No. UNPYSCT-2017091), Supported by the Fundamental Research Foundation for Universities of Heilongjiang Province (LGYC2018JC027).

Author information

Authors and Affiliations

  1. School of Electrical and Electronic Engineering, Harbin University of Science and Technology, Harbin, 150080, China

    Jinbao Xie, Yujing Wang, Qingyan Wang, Baiwei Li & Shiwei Lv

  2. China Telecom Corporation Limited, Harbin, 150080, China

    Yongjin Hou

  3. Department of Telecommunications and Information Technology, Belarusian State University, Minsk, 220030, Belarus

    Yury I. Vorotnitsky

Authors
  1. Jinbao Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yongjin Hou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yujing Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Qingyan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Baiwei Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Shiwei Lv
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Yury I. Vorotnitsky
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

The presented work was carried out in collaboration of all authors. JX put forward ideas, YH conceived and designed the experiment, YH conceived and designed the core model and experiment, created and wrote the paper, YW designed the program, QW was responsible for model parameter debugging, BL analyzed the experimental data and sorted out the data, SL modified and checked the data, and YV proofread the paper.

Corresponding authors

Correspondence to Yongjin Hou or Yujing Wang.

Ethics declarations

Conflict of interest

All authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xie, J., Hou, Y., Wang, Y. et al. Chinese text classification based on attention mechanism and feature-enhanced fusion neural network. Computing 102, 683–700 (2020). https://doi.org/10.1007/s00607-019-00766-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00607-019-00766-9

Keywords

Mathematics Subject Classification

Search

Navigation