Abstract
Knowledge graphs have shown increasing importance in broad applications such as question answering, web search, and recommendation systems. The objective of knowledge graph embedding is to encode both entities and relations of knowledge graphs into continuous low-dimensional vector spaces to perform various machine learning tasks. Most of the existing works only focused on the local structure of knowledge graphs when utilizing structural information of entities, which may not sincerely preserve the global structure of knowledge graphs.In this paper, we propose a semi-supervised model by adopting graph convolutional networks to utilize both local and global structural information of entities. Specifically, our model takes textual information of each entity into consideration as entity attributes in the process of learning. We show the effectiveness of our model by applying it to two traditional tasks for knowledge graph: entity classification and link prediction. Experimental results on two well-known corpora reveal the advantages of this model compared to state-of-the-art methods on both tasks. Moreover, the results show that even with only 1% labeled data to train, our model can still achieve good performance.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Belkin M, Niyogi P (2003) Laplacian eigenmaps for dimensionality reduction and data representation. Neural Comput 15(6):1373–1396
Belkin M, Niyogi P, Sindhwani V (2006) Manifold regularization: a geometric framework for learning from labeled and unlabeled examples. J Mach Learn Res 7(1):2399–2434
Bishop CM (2006) Pattern recognition and machine learning. Springer, Berlin
Bordes A, Usunier N, Garcia-Duran A, Weston J, Yakhnenko O (2013) Translating embeddings for modeling multi-relational data. In: Proceedings of the 26th international conference on neural information processing systems, pp 2787–2795
Bordes A, Glorot Xavier, Weston J, Bengio Y (2014) A semantic matching energy function for learning with multi-relational data. Mach Learn 94(2):233–259
Defferrard M, Bresson X, Vandergheynst P (2016) Convolutional neural networks on graphs with fast localized spectral filtering. In: Proceedings of the 29th international conference on neural information processing systems, pp 1–9
Huang C, Zhu J, Huang X, Yang M, Fung G, Hu Q (2018) A novel approach for entity resolution in scientific documents using context graphs. Inf Sci 432(1):431–441
Ji GL, Liu K, He S, Zhao J (2016) Knowledge graph completion with adaptive sparse transfer matrix. In: Proceedings of the 30th international conference on artificial intelligence, pp 985–991
Ji GL, He SZ, Xu LH, Liu K, Zhao J (2019) Knowledge graph embedding via dynamic mapping matrix. In: Proceedings of the 53th international conference on association for computational linguistics, pp 687–696
Kingma D, Adam J Ba (2015) ADAM: A method for stochastic optimization. In: Proceedings of the 3rd international conference on learning representions, pp 1–15
Kipf TN, Welling M (2016) Variational graph auto-encoders. ArXiv e-prints:1611.07308
Kipf TN, Welling M (2017) Semi-supervised classification with graph convolutional networks. In: Proceedings of the 5th international conference on learning representions, pp 1–14
Lai SW, Xu LH, Liu K, Zhao J (2015) Recurrent convolutional neural networks for text classification. In: Proceedings of the 29th international conference on artificial intelligence, pp 2267–2273
Lao N, Subramanya A, Pereira F, Cohen W (2012) Reading the web with learned syntactic-semantic inference rules. In: Proceedings of the joint conference on empirical methods in natural language processing and computational natural language learning, pp 1017–1026
Li QM, Han ZC, Wu XM (2018) Deeper insights into graph convolutional networks for semi-supervised learning. In: Proceedings of the 32nd international conference on artificial intelligence, pp 1–9
Liben-Nowell D, Kleinberg J (2007) The link-prediction problem for social networks. J Am Soc Inf Sci Technol 58(7):1019–1031
Lin Y, Liu Z, Sun M, Liu Y, Zhu X (2015) Learning entity and relation embeddings for knowledge graph completion. In: Proceedings of the 29th international conference on artificial intelligence, pp 2181–2187
Nickel M, Rosasco L, Poggio T (2016) Holographic embeddings of knowledge graphs. In: Proceedings of the 30th international conference on artificial intelligence, pp 1955–1961
Perozzi B, Al-Rfou R, Skiena S (2014) Deepwalk: online learning of social representations. In: Proceedings of the 20th ACM SIGKDD international conference on knowledge discovery and data mining, pp 701–710
Shi BX, Weninger T (2018) Open-world knowledge graph completion. In: Proceedings of the 32nd international conference on artificial intelligence, pp 1957–1964
Socher R, Chen DQ, Manning C, Ng A (2013) Reasoning with neural tensor networks for knowledge base completion. In: Proceedings of the 26th international conference on neural information processing systems, pp 926–934
Szumlanski S, Gomez F (2010) Automatically acquiring a semantic network of related concepts. In: Proceedings of conference on information and knowledge management, pp 19–28
Tang J, Meng Q, Wang M, Zhang M, Yan J, Mei Q (2015) Line: large-scale information network embedding. In: Proceedings of the 24th international conference on World Wide Web, pp 1067–1077
Toutanova K, Chen D, Pantel P, Choudhury P, Gamon M (2015) Representing text for joint embedding of text and knowledge bases. In: Proceedings of conference on empirical methods in natural language processing, pp 21–28
Velickovic P, Cucurull G, Casanova A, Romero A, Liò P, Bengio Y (2018) Graph attention networks. In: Proceedings of the 6th international conference on learning representions, pp 1–12
Vries GKD (2013) A fast approximation of the weisfeiler-lehman graph kernel for rdf data. In: Proceedings of the 12th international conference on European conference on machine learning and knowledge discovery in databases, pp 606–621
Wang Z, Zhang JW, Feng JL, Chen Z (2014) Knowledge graph and text jointly embedding. In: Proceedings of conference on empirical methods in natural language processing, pp 1591–1601
Wang ZG, Li JZ (2016) Text-enhanced representation learning for knowledge graph. In: Proceedings of the 25th international joint conference on artificial intelligence, pp 1293–1299
Wang D, Cui P, Zhu W (2016) Structural deep network embedding. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp 1225–1234
Wang Q, Mao ZD, Wang B, Guo L (2017) Knowledge graph embedding: a survey of approaches and applications. IEEE Trans Knowl Data Eng 29(12):2724–2743
Wang ZC, Lv QS, Lan XH, Zhang Y (2018) Cross-lingual knowledge graph alignment via graph convolutional networks. In: Proceedings of conference on empirical methods in natural language processing, pp 349–357
Weston J, Collobert R (2008) Deep learning via semi-supervised embedding. In: Proceedings of the 25th international conference on machine learning, pp 1168–1175
Xiao H, Huang ML, Zhu XY (2016) From one point to a manifold: knowledge graph embedding for precise link prediction. In: Proceedings of the 25th international joint conference on artificial intelligence, pp 1315–1321
Xie RB, Liu ZY, Luan HB, Jia J, Sun MS (2016) Representation learning of knowledge graphs with entity descriptions. In: Proceedings of the 30th international conference on artificial intelligence, pp 2659–2665
Xu JC, Qiu XP, Chen K, Huang XJ (2017) Knowledge graph representation with jointly structural and textual encoding. In: Proceedings of the 26th international joint conference on artificial intelligence, pp 1318–1324
Yao L, Mao CS, Luo Y (2019) Graph convolutional networks for text classification. In: Proceedings of the 33rd international conference on artificial intelligence, pp 1–9
Zhong H, Zhang JW, Wang Z, Wan H, Chen Z (2015) Aligning knowledge and text embeddings by entity description. In: Proceedings of conference on empirical methods in natural language processing, pp 267–272
Zhou DY, Bousquet O, Lal TN, Weston J, Scholkopf B (2003) Learning with local and global consistency. In: Proceedings of the 16th international conference on neural information processing systems, pp 321–328
Zhu X, Ghahramani ZB, Lafferty J (2003) Semi-supervised learning using gaussian fields and harmonic functions. In: Proceedings of the 20th international conference on machine learning, pp 912–919
Zhu J, Xie Q, Zheng K (2015) An improved early detection method of type-2 diabetes mellitus using multiple classifier system. Inf Sci 292:1–14
Zhu J, Xie Q, Yu SI, Wong WH (2016) Exploiting link structure for web page genre identification. Data Min Knowl Discov 30(3):550–575
Acknowledgements
This work is supported by the Guangzhou Key Laboratory of Big Data and Intelligent Education (201905010009) and the National Natural Science Foundation of China (Nos. 61877020, 61772211).
Author information
Authors and Affiliations
Corresponding authors
Additional information
Responsible editor: Shuiwang Ji.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Zhu, J., Zheng, Z., Yang, M. et al. A semi-supervised model for knowledge graph embedding. Data Min Knowl Disc 34, 1–20 (2020). https://doi.org/10.1007/s10618-019-00653-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10618-019-00653-z