Recently, link prediction algorithms based on neural embeddings have gained tremendous popularity in the Semantic Web community, and are extensively used for knowledge graph completion. While algorithmic advances have strongly focused on efficient ways of learning embeddings, fewer attention has been drawn to the different ways their performance and robustness can be evaluated. In this work we propose an open-source evaluation pipeline, which benchmarks the accuracy of neural embeddings in situations where knowledge graphs may experience semantic and structural changes. We define relation-centric connectivity measures that allow us to connect the link prediction capacity to the structure of the knowledge graph. Such an evaluation pipeline is especially important to simulate the accuracy of embeddings for knowledge graphs that are expected to be frequently updated.

近年来，基于神经嵌入的链接预测算法在语义Web社区中获得了极大的普及，并被广泛用于知识图的完善。尽管算法的发展一直集中在学习嵌入的有效方法上，但对可评估其性能和鲁棒性的不同方法的关注却很少。在这项工作中，我们提出了一个开源评估管道，该管道在知识图可能经历语义和结构变化的情况下，对神经嵌入的准确性进行了基准测试。我们定义了以关系为中心的连通性度量，这些度量允许我们将链接预测能力连接到知识图的结构。