Semi-supervised learning on graphs is an important problem in the machine learning area. In recent years, state-of-the-art classification methods based on graph neural networks (GNNs) have shown their superiority over traditional ones such as label propagation. However, the sophisticated architectures of these neural models will lead to a complex prediction mechanism, which could not make full use of valuable prior knowledge lying in the data, e.g., structurally correlated nodes tend to have the same class. In this paper, we propose a framework based on knowledge distillation to address the above issues. Our framework extracts the knowledge of an arbitrary learned GNN model (teacher model), and injects it into a well-designed student model. The student model is built with two simple prediction mechanisms, i.e., label propagation and feature transformation, which naturally preserves structure-based and feature-based prior knowledge, respectively. In specific, we design the student model as a trainable combination of parameterized label propagation and feature transformation modules. As a result, the learned student can benefit from both prior knowledge and the knowledge in GNN teachers for more effective predictions. Moreover, the learned student model has a more interpretable prediction process than GNNs. We conduct experiments on five public benchmark datasets and employ seven GNN models including GCN, GAT, APPNP, SAGE, SGC, GCNII and GLP as the teacher models. Experimental results show that the learned student model can consistently outperform its corresponding teacher model by 1.4% - 4.7% on average. Code and data are available at https://github.com/BUPT-GAMMA/CPF

中文翻译：

---

提取图神经网络知识并超越它：一个有效的知识提取框架

图上的半监督学习是机器学习领域中的一个重要问题。近年来，基于图神经网络（GNN）的最新分类方法已显示出其优于传统方法（如标签传播）的优势。但是，这些神经模型的复杂体系结构将导致复杂的预测机制，该机制无法充分利用数据中的宝贵先验知识，例如，结构相关的节点往往具有相同的类别。在本文中，我们提出了一个基于知识提炼的框架来解决上述问题。我们的框架提取了任意学习的GNN模型（教师模型）的知识，并将其注入精心设计的学生模型中。学生模型是通过两种简单的预测机制构建的，即 标签传播和特征转换，自然分别保留了基于结构和基于特征的先验知识。具体来说，我们将学生模型设计为参数化标签传播和特征转换模块的可训练组合。结果，学习的学生可以从先验知识和GNN教师的知识中受益，以获得更有效的预测。此外，与GNN相比，学习型学生模型具有更可解释的预测过程。我们在五个公开基准数据集上进行了实验，并采用了七个GNN模型（包括GCN，GAT，APPNP，SAGE，SGC，GCNII和GLP）作为教师模型。实验结果表明，学习型学生模型平均可以持续优于其相应的教师模型1.4％-4.7％。代码和数据可在https：// github上获得。