Large, heterogeneous datasets are characterized by missing or even erroneous information. This is more evident when they are the product of community effort or automatic fact extraction methods from external sources, such as text. A special case of the aforementioned phenomenon can be seen in knowledge graphs, where this mostly appears in the form of missing or incorrect edges and nodes. Structured querying on such incomplete graphs will result in incomplete sets of answers, even if the correct entities exist in the graph, since one or more edges needed to match the pattern are missing. To overcome this problem, several algorithms for approximate structured query answering have been proposed. Inspired by modern Information Retrieval metrics, these algorithms produce a ranking of all entities in the graph, and their performance is further evaluated based on how high in this ranking the correct answers appear. In this work we take a critical look at this way of evaluation. We argue that performing a ranking-based evaluation is not sufficient to assess methods for complex query answering. To solve this, we introduce Message Passing Query Boxes (MPQB), which takes binary classification metrics back into use and shows the effect this has on the recently proposed query embedding method MPQE.

中文翻译：

---

近似知识图查询答案：从排名到二进制分类

大型，异构数据集的特征在于信息丢失甚至错误。当它们是社区努力或从外部来源（例如文本）自动提取事实的产品时，这一点就更加明显。在知识图中可以看到上述现象的一种特殊情况，其中这种现象大多以缺少或不正确的边和节点的形式出现。即使在图中存在正确的实体，对此类不完整图进行结构化查询也会导致答案集不完整，因为缺少匹配模式所需的一个或多个边。为了克服这个问题，已经提出了几种用于近似结构化查询应答的算法。受现代信息检索指标的启发，这些算法可对图中的所有实体进行排名，并根据出现的正确答案进一步评估他们的表现。在这项工作中，我们对这种评估方式进行了批判性研究。我们认为，执行基于排名的评估还不足以评估复杂查询回答的方法。为了解决这个问题，我们引入了消息传递查询框（Message Passing Query Boxs，MPQB），该框重新使用了二进制分类指标，并显示了其对最近提出的查询嵌入方法MPQE的影响。