Skip to main content
SpringerLink
Log in

Hierarchical LSTM with char-subword-word tree-structure representation for Chinese named entity recognition

  • Research Paper
  • Published:
Science China Information Sciences Aims and scope Submit manuscript

Abstract

Chinese named entity recognition (CNER) aims to identify entity names such as person names and organization names from Chinese raw text and thus can quickly extract the entity information that people are concerned about from large-scale texts. Recent studies attempt to improve performance by integrating lexicon words into char-based CNER models. These existing studies, however, usually focus on leveraging the context-free words in lexicon without considering the contextual information of words and subwords in the sentences. To address this issue, in addition to utilizing the lexicon words, we further propose to construct a hierarchical tree structure representation composed of characters, subwords and context-aware predicted words from segmentor to represent each sentence for CNER. Based on the tree-structure representation, we propose a hierarchical long short-term memory (HiLSTM) framework, which consists of hierarchical encoding layer, fusion layer and CRF layer, to capture linguistic knowledge at different levels. On the one hand, the interactions within each level help to obtain the contextual information. On the other hand, the propagations from the lower-levels to the upper-levels can provide additional semantic knowledge for CNER. Experimental results on three widely used CNER datasets show that our proposed HiLSTM model achieves significant improvement over several strong benchmark methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Chen Y B, Xu L H, Liu K, et al. Event extraction via dynamic multi-pooling convolutional neural networks. In: Proceedings of Annual Meeting of the Association for Computational Linguistics (ACL), 2015. 167–176

  2. Miwa M, Bansal M. End-to-end relation extraction using LSTMs on sequences and tree structures. In: Proceedings of Annual Meeting of the Association for Computational Linguistics (ACL), 2016. 1105–1116

  3. Diefenbach D, Lopez V, Singh K, et al. Core techniques of question answering systems over knowledge bases: a survey. Knowl Inform Syst, 2018. https://hal.archives-ouvertes.fr/hal-01637143/document

  4. Yang J F, Guan Y, He B, et al. Corpus construction for named entities and entity relations on Chinese electronic medical records. J Softw, 2016, 27: 2725–2746

    Google Scholar 

  5. Song L F, Zhang Y, Gildea D, et al. Leveraging dependency forest for neural medical relation extraction. In: Proceedings of Empirical Methods in Natural Language Processing (EMNLP), 2019. 208–218

  6. Tian Y H, Ma W C, Xia F, et al. ChiMed: a Chinese medical corpus for question answering. In: Proceedings of the 18th BioNLP Workshop and Shared Task, 2019. 250–260

  7. Saito K, Nagata M. Multi-language named-entity recognition system based on HMM. In: Proceedings of Annual Meeting of the Association for Computational Linguistics Workshop on Multilingual and Mixed-language Named Entity Recognition, 2003

  8. Yu H K, Zhang H P, Liu Q, et al. Chinese named entity identification using cascaded hidden Markov model. J Commun, 2006, 27: 87–94

    Google Scholar 

  9. Solorio T, López A. Learning named entity classifiers using support vector machines. In: Proceedings of Conference on Computational Linguistics and Natural Language Processing (CICLing), 2004. 158–167

  10. Mccallum A, Li W. Early results for named entity recognition with conditional random fields, feature induction and web-enhanced lexicons. In: Proceedings of Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics (HLT-NAACL), 2003. 188–191

  11. Chiu J P C, Nichols E. Named entity recognition with bidirectional LSTM-CNNs. Trans Assoc Comput Linguist, 2016, 4: 357–370

    Article  Google Scholar 

  12. Lample G, Ballesteros M, Subramanian S, et al. Neural architectures for named entity recognition. In: Proceedings of Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics (HLT-NAACL), 2016. 260–270

  13. Liu L Y, Shang J B, Xu F, et al. Empower sequence labeling with task-aware neural language model. In: Proceedings of Association for the Advance of Artificial Intelligence (AAAI), 2018. 5253–5260

  14. Dong C H, Zhang J J, Zong C Q, et al. Character-based LSTM-CRF with radical-level features for Chinese named entity recognition. In: Proceedings of International Conference on Computer Processing of Oriental Languages (ICCPOL), 2016. 239–250

  15. Zhang Y, Yang J. Chinese NER using lattice LSTM. In: Proceedings of Annual Meeting of the Association for Computational Linguistics (ACL), 2018. 1554–1564

  16. Gui T, Zou Y C, Zhang Q, et al. A lexicon-based graph neural network for Chinese NER. In: Proceedings of Empirical Methods in Natural Language Processing (EMNLP), 2019. 1040–1050

  17. Liu W, Xu T G, Xu Q H, et al. An encoding strategy based word-character LSTM for Chinese NER. In: Proceedings of North American Chapter of the Association for Computational Linguistics (NAACL), 2019. 2379–2389

  18. Sui D B, Chen Y B, Liu K, et al. Leverage lexical knowledge for Chinese named entity recognition via collaborative graph network. In: Proceedings of Empirical Methods in Natural Language Processing (EMNLP), 2019. 3828–3838

  19. Gong C, Li Z H, Zhang M, et al. Multi-grained Chinese word segmentation. In: Proceedings of Empirical Methods in Natural Language Processing (EMNLP), 2017. 703–714

  20. Heinzerling B, Strube M. BPEmb: tokenization-free pre-trained subword embeddings in 275 languages. In: Proceedings of the International Conference on Language Resources and Evaluation (LREC), 2018. 2989–2993

  21. Collobert R, Weston J, Bottou L, et al. Natural language processing (almost) from scratch. J Mach Learn Res, 2011, 12: 2493–2537

    MATH  Google Scholar 

  22. He H F, Sun X. A unified model for cross-domain and semi-supervised named entity recognition in Chinese social media. In: Proceedings of Association for the Advance of Artificial Intelligence (AAAI), 2017. 3216–3222

  23. Zhao H, Kit C Y. Unsupervised segmentation helps supervised learning of character tagging for word segmentation and namedentity recognition. In: Proceedings of SIGHAN Workshop on Chinese Language Processing, 2008

  24. Peng N Y, Dredze M. Named entity recognition for Chinese social media with jointly trained embeddings. In: Proceedings of Empirical Methods in Natural Language Processing (EMNLP), 2015. 548–554

  25. He H F, Sun X. F-score driven max margin neural network for named entity recognition in Chinese social media. In: Proceedings of European Chapter of the Association for Computational Linguistics (EACL), 2017. 713–718

  26. Peng N Y, Dredze M. Improving named entity recognition for Chinese social media with word segmentation representation learning. In: Proceedings of Annual Meeting of the Association for Computational Linguistics (ACL), 2016. 149–155

  27. Cao P F, Chen Y B, Liu K, et al. Adversarial transfer learning for Chinese named entity recognition with self-attention mechanism. In: Proceedings of Empirical Methods in Natural Language Processing (EMNLP), 2018. 182–192

  28. Sennrich R, Haddow B, Birch A. Neural machine translation of rare words with subword units. In: Proceedings of Annual Meeting of the Association for Computational Linguistics (ACL), 2016. 1715–1725

  29. Weischedel R, Palmer M, Marcus M, et al. OntoNotes Release 4.0. Philadelphia: Linguistic Data Consortium, 2011

  30. Levow G A. The third international Chinese language processing backoff: word segmentation and named entity recognition. In: Proceedings of SIGHAN Workshop on Chinese Language Processing, 2006. 108–117

  31. Noreen E. Computer-intensive methods for testing hypotheses. Biometrics, 1990, 46: 540–541

    Google Scholar 

  32. Zhang S X, Qin Y, Wen J, et al. Word segmentation and named entity recognition for SIGHAN bakeoff3. In: Proceedings of SIGHAN Workshop on Chinese Language Processing, 2006. 158–161

  33. Zhou J S, Qu W G, Zhang F. Chinese named entity recognition via joint identification and categorization. Chinese J Electron, 2013, 22: 225–230

    Google Scholar 

  34. Li S, Zhao Z, Hu R F, et al. Analogical reasoning on Chinese morphological and semantic relations. In: Proceedings of Annual Meeting of the Association for Computational Linguistics (ACL), 2018. 138–143

Download references

Acknowledgements

This work was supported by National Science Fund for Distinguished Young Scholars (Grant No. 61525205), National Natural Science Foundation of China (Grant No. 61876116), and Project Funded by the Priority Academic Program Development of Jiangsu Higher Education Institutions.

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, Soochow University, Suzhou, 215006, China

    Chen Gong, Zhenghua Li, Qingrong Xia, Wenliang Chen & Min Zhang

Authors
  1. Chen Gong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhenghua Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Qingrong Xia
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Wenliang Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Min Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhenghua Li.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gong, C., Li, Z., Xia, Q. et al. Hierarchical LSTM with char-subword-word tree-structure representation for Chinese named entity recognition. Sci. China Inf. Sci. 63, 202102 (2020). https://doi.org/10.1007/s11432-020-2982-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11432-020-2982-y

Keywords

Search

Navigation