In this paper, we present a semi-automatic system (Sherlock) for quiz generation using linked data and textual descriptions of RDF resources. Sherlock is distinguished from existing quiz generation systems in its generic framework for domain-independent quiz generation as well as in the ability of controlling the difficulty level of the generated quizzes. Difficulty scaling is non-trivial, and it is fundamentally related to cognitive science. We approach the problem with a new angle by perceiving the level of knowledge difficulty as a similarity measure problem and propose a novel hybrid semantic similarity measure using linked data. Extensive experiments show that the proposed semantic similarity measure outperforms four strong baselines with more than 47 % gain in clustering accuracy. In addition, we discovered in the human quiz test that the model accuracy indeed shows a strong correlation with the pairwise quiz similarity.

中文翻译：

---

Sherlock：使用混合语义相似性度量的测验生成半自动框架。


在本文中，我们提出了一个使用链接数据和 RDF 资源文本描述生成测验的半自动系统 (Sherlock)。 Sherlock 与现有测验生成系统的区别在于其用于与领域无关的测验生成的通用框架以及控制所生成测验的难度级别的能力。难度缩放并非微不足道，它与认知科学有着根本的关系。我们通过将知识难度水平视为相似性度量问题，以新的角度来处理该问题，并提出了一种使用链接数据的新型混合语义相似性度量。大量实验表明，所提出的语义相似性度量优于四个强大的基线，聚类精度提高了 47% 以上。此外，我们在人类测验测试中发现，模型准确性确实与成对测验相似度表现出很强的相关性。