During the years, it has been possible to assess significant improvements in the computational efficiency of Semantic Web search and exploration systems. However, it has been much harder to assess how well different semantic systems’ user interfaces help their users. One of the key factors facilitating the advancement of research in a particular field is the ability to compare the performance of different approaches. Though there are many such benchmarks in Semantic Web fields that have experienced significant improvements, this is not the case for Semantic Web user interfaces for data exploration. We propose and demonstrate the use of a benchmark for evaluating such user interfaces, which includes a set of typical user tasks and a well-defined procedure for assigning a measure of performance on those tasks to a semantic system. We have applied the benchmark to four such systems. Moreover, all the required resources to apply the benchmark are openly available online. We intend to initiate a community conversation that will lead to a generally accepted framework for comparing systems and for measuring, and thus encouraging, progress towards better semantic search and exploration tools.

这些年来，有可能评估语义Web搜索和探索系统在计算效率方面的显着改进。但是，要评估不同语义系统的用户界面对用户的帮助程度要困难得多。促进特定领域研究进展的关键因素之一是能够比较不同方法的性能。尽管语义Web领域中有许多这样的基准已经取得了重大改进，但是用于数据探索的语义Web用户界面却并非如此。我们提出并演示了使用基准来评估此类用户界面，其中包括一组典型的用户任务以及一个明确定义的过程，用于将这些任务的性能度量分配给语义系统。我们已将基准应用到四个此类系统。此外，应用基准测试所需的所有资源均可在线在线获得。我们打算发起一个社区对话，这将导致一个公认的框架，该框架用于比较系统，衡量和鼓励发展更好的语义搜索和探索工具。