Skip to main content

Advertisement

Log in

Named entity recognition for Chinese marine text with knowledge-based self-attention

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Chinese named entity recognition has been widely used in many fields, such as species recognition in marine information, and so on. Compared with the standard named entity recognition (NER), the performance of the Chinese marine named entity recognition is low, which is mainly limited by the normative nature of the text and the scale of the tagged corpus. In recent years, the research of named entity recognition primarily focuses on the small scale of the tagged corpus. It tends to use external knowledge or use joint training to improve final recognition performance. However, there is little research on the problem that the accuracy of NER will be suboptimal if the model is trained inadequately. To this end, in order to improve the accuracy of naming entity recognition and recognize more entities in corpus, this paper proposes a named entity recognition method that combines knowledge graph embedding with a self-attention mechanism. The entity embeddings of the marine knowledge graph (KG) are empolyed for the hidden units of NER model with attention mechanism in an end-to-end way. Therefore, the model can get additional auxiliary information to improve performance. Lastly, we conduct extensive experiments on marine corpus and other public datasets. The experimental results verify the effectiveness of our proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Bordes A, Usunier N, Garcia-Duran A, Weston J, Yakhnenko O (2013) Translating embeddings for modeling multi-relational data. In: Advances in neural information processing systems, pp 2787–2795

  2. Bunescu R C, Mooney R J (2005) A shortest path dependency kernel for relation extraction. In: EMNLP, pp 724–731

  3. Cao P, Chen Y, Liu K, Zhao J, Liu S (2018) Adversarial transfer learning for chinese named entity recognition with self-attention mechanism. In: Proceedings of the 2018 conference on empirical methods in natural language processing, pp 182–192

  4. Chen L, Yang M (2017) Semi-supervised dictionary learning with label propagation for image classification. Comput Vis Media 3(1):83–94

    Article  Google Scholar 

  5. Chen Y, Xu L, Liu K, Zeng D, Zhao J (2015) Event extraction via dynamic multi-pooling convolutional neural networks. In: ACL, pp 167–176

  6. Chorowski J K, Bahdanau D, Serdyuk D, Cho K, Bengio Y (2015) Attention-based models for speech recognition. In: Advances in neural information processing systems, pp 577–585

  7. Devlin J, Chang M -W, Lee K, Toutanova K (2018) Bert: pre-training of deep bidirectional transformers for language understanding. arXiv:1810.04805

  8. Ebisu T, Ichise R (2018) Toruse: knowledge graph embedding on a lie group. In: Thirty-second AAAI conference on artificial intelligence

  9. Fader A, Zettlemoyer L, Etzioni O (2013) Paraphrase-driven learning for open question answering. In: ACL, pp 1608–1618

  10. Fan D -P, Cheng M -M, Liu J -J, Gao S -H, Hou Q, Borji A (2018) Salient objects in clutter: bringing salient object detection to the foreground. In: Proceedings of the European conference on computer vision (ECCV), pp 186–202

  11. Forney G D (1973) The viterbi algorithm. Proc IEEE 61(3):268–278

    Article  MathSciNet  Google Scholar 

  12. Fu K, Zhao Q, Gu I Y -H, Yang J (2019) Deepside: a general deep framework for salient object detection. Neurocomputing 356:69–82

    Article  Google Scholar 

  13. Fu K, Fan D -P, Ji G -P, Zhao Q (2020) Jl-dcf: joint learning and densely-cooperative fusion framework for rgb-d salient object detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 3052–3062

  14. Greenberg N, Bansal T, Verga P, McCallum A (2018) Marginal likelihood training of bilstm-crf for biomedical named entity recognition from disjoint label sets. In: Proceedings of the 2018 conference on empirical methods in natural language processing, pp 2824–2829

  15. Guo S, Wang Q, Wang L, Wang B, Guo L (2018) Knowledge graph embedding with iterative guidance from soft rules. In: Thirty-second AAAI conference on artificial intelligence

  16. Han S, Hao X, Huang H (2018) An event-extraction approach for business analysis from online chinese news. Electron Commer Res Appl 28:244–260

    Article  Google Scholar 

  17. He H, Sun X (2016) F-score driven max margin neural network for named entity recognition in chinese social media. arXiv:1611.04234

  18. He H, Sun X (2017) A unified model for cross-domain and semi-supervised named entity recognition in Chinese social media. In: Thirty-first AAAI conference on artificial intelligence

  19. Hochreiter S, Schmidhuber J (1997) Lstm can solve hard long time lag problems. In: Advances in neural information processing systems, pp 473–479

  20. Huang Z, Xu W, Yu K (2015) Bidirectional lstm-crf models for sequence tagging. arXiv:1508.01991

  21. Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. arXiv:1502.03167

  22. Ju M, Miwa M, Ananiadou S (2018) A neural layered model for nested named entity recognition. In: Proceedings of the 2018 conference of the North American Chapter Of The Association For Computational Linguistics: human language technologies, vol 1 (Long Papers), pp 1446–1459

  23. Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. arXiv:1412.6980

  24. Lafferty J, McCallum A, Pereira F C (2001) Conditional random fields: probabilistic models for segmenting and labeling sequence data

  25. Lample G, Ballesteros M, Subramanian S, Kawakami K, Dyer C (2016) Neural architectures for named entity recognition. In: Proceedings of the 2016 conference of the North American chapter of the association for computational linguistics: human language technologies, pp 260–270

  26. Lee D, Yu W, Lim H (2017) Bi-directional lstm-cnn-crf for korean named entity recognition system with feature augmentation. J Korea Converg Soc 8(12):55–62

    Google Scholar 

  27. Lei Ba J, Kiros J R, Hinton G E (2016) Layer normalization. arXiv:1607.06450

  28. Lin Y, Liu Z, Sun M, Liu Y, Zhu X (2015) Learning entity and relation embeddings for knowledge graph completion. In: Twenty-ninth AAAI conference on artificial intelligence

  29. Liu W, Xu T, Xu Q, Song J, Zu Y (2019) An encoding strategy based word-character lstm for Chinese ner. In: Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, vol 1 (Long and Short Papers), pp 2379–2389

  30. Lu X, Wang W, Ma C, Shen J, Shao L, Porikli F (2019) See more, know more: unsupervised video object segmentation with co-attention siamese networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3623–3632

  31. Lu X, Wang W, Shen J, Tai Y -W, Crandall D J, Hoi S C (2020) Learning video object segmentation from unlabeled videos. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 8960–8970

  32. McClosky D, Surdeanu M, Manning C D (2011) Event extraction as dependency parsing. In: HLT, pp 1626–1635

  33. Mikolov T, Sutskever I, Chen K, Corrado G S, Dean J (2013) Distributed representations of words and phrases and their compositionality. In: Advances in neural information processing systems, pp 3111–3119

  34. Miwa M, Bansal M (2016) End-to-end relation extraction using lstms on sequences and tree structures. In: ACL, pp 1105–1116

  35. Ng A Y (2004) Feature selection, l 1 vs. l 2 regularization, and rotational invariance. In: Proceedings of the twenty-first international conference on machine learning. ACM, p 78

  36. Park G, Kim H (2018) Low-cost implementation of a named entity recognition system for voice-activated human-appliance interfaces in a smart home. Sustainability 10(2):488

    Article  Google Scholar 

  37. Peng N, Dredze M (2015) Named entity recognition for chinese social media with jointly trained embeddings. In: Proceedings of the 2015 conference on empirical methods in natural language processing, pp 548–554

  38. Peng N, Dredze M (2016) Improving named entity recognition for chinese social media with word segmentation representation learning. arXiv:1603.00786

  39. Peng D, Wang Y, Liu C, Chen Z (2019) Tl-ner: a transfer learning model for chinese named entity recognition. Information Systems Frontiers 1–14

  40. Pennington J, Socher R, Manning C (2014) Glove: global vectors for word representation. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pp 1532–1543

  41. Singh S, Riedel S, Martin B, Zheng J, McCallum A (2013) Joint inference of entities, relations, and coreference. In: AKBC, pp 1–6

  42. Upadhyay S, Gupta N, Roth D (2018) Joint multilingual supervision for cross-lingual entity linking. In: EMNLP, pp 2486–2495

  43. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez A N, Kaiser Ł, Polosukhin I (2017) Attention is all you need. In: Advances in neural information processing systems, pp 5998–6008

  44. Voorhees E, Harman D K (2006) Trec : experiment and evaluation in information retrieval. J Am Soc Inf Sci Technol 32(4):563–567

    Google Scholar 

  45. Wang Z, Zhang J, Feng J, Chen Z (2014) Knowledge graph embedding by translating on hyperplanes. In: Twenty-Eighth AAAI conference on artificial intelligence

  46. Wang Q, Mao Z, Wang B, Guo L (2017) Knowledge graph embedding: a survey of approaches and applications. IEEE Trans Knowl Data Eng 29 (12):2724–2743

    Article  Google Scholar 

  47. Wang W, Lu X, Shen J, Crandall D J, Shao L (2019) Zero-shot video object segmentation via attentive graph neural networks. In: Proceedings of the IEEE international conference on computer vision, pp 9236–9245

  48. Xiang Y, et al. (2017) Chinese named entity recognition with character-word mixed embedding. In: Proceedings of the 2017 ACM on conference on information and knowledge management. ACM, pp 2055–2058

  49. Xiong C, Power R, Callan J (2017) Explicit semantic ranking for academic search via knowledge graph embedding. In: Proceedings of the 26th international conference on world wide web, pp 1271–1279

  50. Xin J, Lin Y, Liu Z, Sun M (2018) Improving neural fine-grained entity typing with knowledge attention. In: AAAI, pp 1–8

  51. Xu C, Wang F, Han J, Li C (2019) Exploiting multiple embeddings for chinese named entity recognition. In: Proceedings of the 28th ACM international conference on information and knowledge management, pp 2269–2272

  52. Xu D, Ruan C, Korpeoglu E, Kumar S, Achan K (2020) Product knowledge graph embedding for e-commerce. In: Proceedings of the 13th international conference on web search and data mining, pp 672–680

  53. Yao X, Van Durme B (2014) Information extraction over structured data: question answering with freebase. In: ACL, pp 956–966

  54. Yao Y, Rosasco L, Caponnetto A (2007) On early stopping in gradient descent learning. Construct Approx 26(2):289–315

    Article  MathSciNet  Google Scholar 

  55. Yao L, Torabi A, Cho K, Ballas N, Pal C, Larochelle H, Courville A (2015) Video description generation incorporating spatio-temporal features and a soft-attention mechanism. arXiv:1502.08029

  56. Yubo C, Liheng X, Kang L, Daojian Z, Jun Z et al (2015) Event extraction via dynamic multi-pooling convolutional neural networks. In: ACL, pp 167–176

  57. Zhang Y, Yang J (2018) Chinese ner using lattice lstm. arXiv:1805.02023

  58. Zhao J -X, Cao Y, Fan D -P, Cheng M -M, Li X -Y, Zhang L (2019) Contrast prior and fluid pyramid integration for rgbd salient object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3927–3936

Download references

Acknowledgments

This work is supported by the Second Level Research Project of China Geological Survey (Grant No. DD20191008).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Shufeng He.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

He, S., Sun, D. & Wang, Z. Named entity recognition for Chinese marine text with knowledge-based self-attention. Multimed Tools Appl 81, 19135–19149 (2022). https://doi.org/10.1007/s11042-020-10089-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-10089-z

Keywords

Navigation