Finding a good query plan is key to the optimization of query runtime. This holds in particular for cost-based federation engines, which make use of cardinality estimations to achieve this goal. A number of studies compare SPARQL federation engines across different performance metrics, including query runtime, result set completeness and correctness, number of sources selected and number of requests sent. Albeit informative, these metrics are generic and unable to quantify and evaluate the accuracy of the cardinality estimators of cost-based federation engines. To thoroughly evaluate cost-based federation engines, the effect of estimated cardinality errors on the overall query runtime performance must be measured. In this paper, we address this challenge by presenting novel evaluation metrics targeted at a fine-grained benchmarking of cost-based federated SPARQL query engines. We evaluate five cost-based federated SPARQL query engines using existing as well as novel evaluation metrics by using LargeRDFBench queries. Our results provide a detailed analysis of the experimental outcomes that reveal novel insights, useful for the development of future cost-based federated SPARQL query processing engines.

中文翻译：

---

基于成本的联合SPARQL查询处理引擎的经验评估

找到一个好的查询计划是优化查询运行时的关键。这尤其适用于基于成本的联合引擎，该引擎使用基数估计来实现此目标。大量研究在不同的性能指标上比较了SPARQL联合引擎，包括查询运行时，结果集的完整性和正确性，选择的源数和发送的请求数。尽管提供了信息，但这些指标是通用的，无法量化和评估基于成本的联合引擎基数估计量的准确性。为了彻底评估基于成本的联合引擎，必须测量估计基数错误对整体查询运行时性能的影响。在本文中，我们通过提出针对基于成本的联合SPARQL查询引擎的细粒度基准测试的新颖评估指标来应对这一挑战。我们通过使用LargeRDFBench查询，使用现有的以及新颖的评估指标来评估五个基于成本的联合SPARQL查询引擎。我们的结果提供了对实验结果的详细分析，揭示了新颖的见解，对开发未来基于成本的联合SPARQL查询处理引擎很有用。