当前位置:
X-MOL 学术
›
arXiv.cs.SI
›
论文详情
Our official English website, www.x-mol.net, welcomes your
feedback! (Note: you will need to create a separate account there.)
Extract the Knowledge of Graph Neural Networks and Go Beyond it: An Effective Knowledge Distillation Framework
arXiv - CS - Social and Information Networks Pub Date : 2021-03-04 , DOI: arxiv-2103.02885 Cheng Yang, Jiawei Liu, Chuan Shi
arXiv - CS - Social and Information Networks Pub Date : 2021-03-04 , DOI: arxiv-2103.02885 Cheng Yang, Jiawei Liu, Chuan Shi
Semi-supervised learning on graphs is an important problem in the machine
learning area. In recent years, state-of-the-art classification methods based
on graph neural networks (GNNs) have shown their superiority over traditional
ones such as label propagation. However, the sophisticated architectures of
these neural models will lead to a complex prediction mechanism, which could
not make full use of valuable prior knowledge lying in the data, e.g.,
structurally correlated nodes tend to have the same class. In this paper, we
propose a framework based on knowledge distillation to address the above
issues. Our framework extracts the knowledge of an arbitrary learned GNN model
(teacher model), and injects it into a well-designed student model. The student
model is built with two simple prediction mechanisms, i.e., label propagation
and feature transformation, which naturally preserves structure-based and
feature-based prior knowledge, respectively. In specific, we design the student
model as a trainable combination of parameterized label propagation and feature
transformation modules. As a result, the learned student can benefit from both
prior knowledge and the knowledge in GNN teachers for more effective
predictions. Moreover, the learned student model has a more interpretable
prediction process than GNNs. We conduct experiments on five public benchmark
datasets and employ seven GNN models including GCN, GAT, APPNP, SAGE, SGC,
GCNII and GLP as the teacher models. Experimental results show that the learned
student model can consistently outperform its corresponding teacher model by
1.4% - 4.7% on average. Code and data are available at
https://github.com/BUPT-GAMMA/CPF
中文翻译:
提取图神经网络知识并超越它:一个有效的知识提取框架
图上的半监督学习是机器学习领域中的一个重要问题。近年来,基于图神经网络(GNN)的最新分类方法已显示出其优于传统方法(如标签传播)的优势。但是,这些神经模型的复杂体系结构将导致复杂的预测机制,该机制无法充分利用数据中的宝贵先验知识,例如,结构相关的节点往往具有相同的类别。在本文中,我们提出了一个基于知识提炼的框架来解决上述问题。我们的框架提取了任意学习的GNN模型(教师模型)的知识,并将其注入精心设计的学生模型中。学生模型是通过两种简单的预测机制构建的,即 标签传播和特征转换,自然分别保留了基于结构和基于特征的先验知识。具体来说,我们将学生模型设计为参数化标签传播和特征转换模块的可训练组合。结果,学习的学生可以从先验知识和GNN教师的知识中受益,以获得更有效的预测。此外,与GNN相比,学习型学生模型具有更可解释的预测过程。我们在五个公开基准数据集上进行了实验,并采用了七个GNN模型(包括GCN,GAT,APPNP,SAGE,SGC,GCNII和GLP)作为教师模型。实验结果表明,学习型学生模型平均可以持续优于其相应的教师模型1.4%-4.7%。代码和数据可在https:// github上获得。
更新日期:2021-03-05
中文翻译:
提取图神经网络知识并超越它:一个有效的知识提取框架
图上的半监督学习是机器学习领域中的一个重要问题。近年来,基于图神经网络(GNN)的最新分类方法已显示出其优于传统方法(如标签传播)的优势。但是,这些神经模型的复杂体系结构将导致复杂的预测机制,该机制无法充分利用数据中的宝贵先验知识,例如,结构相关的节点往往具有相同的类别。在本文中,我们提出了一个基于知识提炼的框架来解决上述问题。我们的框架提取了任意学习的GNN模型(教师模型)的知识,并将其注入精心设计的学生模型中。学生模型是通过两种简单的预测机制构建的,即 标签传播和特征转换,自然分别保留了基于结构和基于特征的先验知识。具体来说,我们将学生模型设计为参数化标签传播和特征转换模块的可训练组合。结果,学习的学生可以从先验知识和GNN教师的知识中受益,以获得更有效的预测。此外,与GNN相比,学习型学生模型具有更可解释的预测过程。我们在五个公开基准数据集上进行了实验,并采用了七个GNN模型(包括GCN,GAT,APPNP,SAGE,SGC,GCNII和GLP)作为教师模型。实验结果表明,学习型学生模型平均可以持续优于其相应的教师模型1.4%-4.7%。代码和数据可在https:// github上获得。