Abstract
Natural language processing is an application of a computational technique that allows the machine to process human language. One of the primary tasks of NLP is information extraction that aims to capture important information from the text. Nowadays, the fast-growing web contains a large amount of textual information, requires a technique to extract relevant information. The entity recognition task is a type of information extraction that attempts to find and classify named entities appearing in the unstructured text document. The traditional coarse-grained entity recognition systems often define a less number of pre-defined named entity categories such as person, location, organization, and date. The fine-grained entity type classification model focused to classify the target entities into fine-grained types. Most of the recent works are accomplished with the help of Bidirectional LSTM with an attention mechanism. But due to the complex structure of bidirectional LSTM, these models consume an enormous amount of time for the training process. The existing attention mechanisms are incapable to pick up the correlation between the new word and the previous context. The proposed system resolves this issue by utilizing bidirectional GRU with the self-attention mechanism. The experiment result shows that the novel approach outperforms state-of-the-art methods.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Collobert R, Waton J (2008) A unified architecture for natural language processing: deep neural networks with multitask learning. In: Proceedings of the 25th international conference on Machine learning, ACM, pp. 160–167
Nadeau D, Sekine S (2007) A survey of named entity recognition and classification. Lingvisticae Investigationes 30(1):3–26
Cowie J, Wilks Y (2000) Information extraction. In: Dale R, Moisl H, Somers HL (eds) Handbook of natural language processing, pp 241–260
Chinchor N, Robinson P (1997) Muc-7 named entity task definition. In: Proceedings of the 7th conference on message understanding, volume 29, pp 1–21
Chinchor N (1998) Overview of muc-7. In: Seventh message understanding conference (MUC-7), Proceedings of a Conference Held in Fairfax, Virginia
Balasuriya D, Ringland N, Nothman J, Murphy T, Curran JR (2009) Named entity recognition in Wikipedia. In: Proceedings of the 2009 Workshop on The People’s Web Meets NLP: Collaboratively Constructed Semantic Resources (People’s Web), pp. 10–18
Fu R, Zhang Z, Li L (2016) Using lstm and gru neural network methods for traffic flow prediction, Youth Academic Annual Conference of Chinese Association of Automation (YAC), IEEE, pp. 324–328
Chung J, Gulcehre C, Cho K, Bengio Y (2014) Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint. arXiv:1412.3555
Dey R, Salemt FM (2017) Gate-variants of gated recurrent unit (gru) neural networks. IEEE 60th international midwest symposium on circuits and systems (MWSCAS), IEEE, pp. 1597–1600
Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser L, Polosukhin I (2017) Attention is all you need. In: Advances in neural information processing systems, pp 5998–6008
Petasis G, Cucchiarelli A, Velardi P, Paliouras G, Karkaletsis V, Spyropoulos CD (2000) Automatic adaptation of proper noun dictionaries through cooperation of machine learning and probabilistic methods. In: 23rd Annual International ACM SIGIR conference on research and development in information retrieval, pp. 128–135
Alfonseca E, Manandhar S (2002) An unsupervised method for general named entity recognition and automated concept discovery. In: 1st International Conference on General WordNet
NIST (2008) Automatic Content Extraction Evaluation (ACE08). Official Results.
Balog K, Serdyukov P, De Vries AP (2010) Overview of the TREC 2010 entity track. Norwegian University of Science and Technology, Trondheim
Demartini G, Iofciu T, De Vries AP (2009) Overview of the INEX 2009 entity ranking track. In: International workshop of the initiative for the evaluation of XML retrieval. Springer, Berlin, pp 254–264
Ji H, Grishman R (2011) Knowledge base population: successful approaches and challenges. In: Annual meeting of the association for computational linguistics, pp. 1148–1158
Lee C, Hwang Y-G, Oh H-J, Lim S, Heo J, Lee C-H, Kim H-J, Wang J-H, Jang M-G (2006) Fine-grained named entity recognition using conditional random fields for question answering, In: asia information retrieval symposium, Springer, pp. 581–587
Ling X, Weld DS (2012) Fine-grained entity recognition. In: Twenty-Sixth AAAI conference on artificial intelligence
Yosef MA, Bauer S, Hoffart J, Spaniol M, Weikum G (2012) Hyena: hierarchical type classification for entity names. Proceedings of COLING 2012: Posters, pp. 1361–1370
Gillick D, Lazic N, Ganchev K, Kirchner J, Huynh D (2014) Context-dependent fine-grained entity type tagging. arXiv preprint. arXiv:1412.1820
Yogatama D, Gillick D, Lazic N (2015) Embedding methods for fine grained entity type classification. In: Proceedings of the 53rd annual meeting of the association for computational linguistics and the 7th international joint conference on natural language processing vol. 2, pp. 291–296
Cui KY, Ren PJ, Chen ZM, Lian T, Ma J (2017) Relation enhanced neural model for type classification of entity mentions with a fine-grained taxonomy. J Comput Sci Technol 32(4):814–827
Shimaoka S, Stenetorp P, Inui K, Riedel S (2016) Neural architectures for fine-grained entity type classification. arXiv preprint. arXiv:1606.01341
Yaghoobzadeh Y, Schütze H (2018) Multi-multi-view learning: multilingual and multi-representation entity typing. arXiv preprint. arXiv:1810.1049
Anand A, Awekar A (2017) Fine-grained entity type classification by jointly learning representations and label embeddings. arXiv preprint. arXiv:1702.06709
Xu P, Barbosa D (2018) Neural fine-grained entity type classification with hierarchy-aware loss. arXiv preprint. arXiv:1803.03378
Liu J, Wang L, Zhou M, Wang J, Lee S (2018) Fine-grained entity type classification with adaptive context. Soft Comput 22(13):4307–4318
Xin J, Lin Y, Liu Z, Sun M (2018) Improving neural fine-grained entity typing with knowledge attention. In: Thirty-second AAAI conference on artificial intelligence
Hovy E, Marcus M, Palmer M, Ramshaw L, Weischedel R (2006) OntoNotes: the 90% solution. In: Proceedings of the human language technology conference of the NAACL, Companion Volume: Short Papers, pp 57–60
Hachey B, Radford W, Nothman J, Honnibal M, Curran JR (2013) Evaluating entity linking with wikipedia. Artif Intell 194:130–150
Kenter T, De Rijke M (2015) Short text similarity with word embeddings. In: Proceedings of the 24th ACM international on conference on information and knowledge management, pp 1411–1420
Pennington J, Socher R, Manning CD (2014) Glove: global vectors for word representation. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pp 1532–1543
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Dhrisya, K., Remya, G. & Mohan, A. Fine-grained entity type classification using GRU with self-attention. Int. j. inf. tecnol. 12, 869–878 (2020). https://doi.org/10.1007/s41870-020-00499-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s41870-020-00499-5