Semantic annotation is the process by which existing texts receive a mark-up that allows automatic identification of named entities (e.g., to distinguish between a turkey bird and the country Turkey). Manual annotation is useful both as stand-alone process in a domain-specific setting or as post-processing for automatic annotation algorithms. Development of most annotation tools strongly focuses on providing novel functionality — end-user evaluations of interface and user interaction are rare.

This article reports on the results of a series of user studies executed to explore how non-expert users understand the process of enriching texts with semantic annotations. We find that these users can easily create simple semantic annotations (e.g., assigning concepts to text passages) but have difficulties understanding complex semantic annotations (e.g., assigning semantic identifiers to text passages). We present the results of three user studies on manual semantic annotation and discusses the lessons learned about both the semantic enrichment process and our methodology of exposing non-experts to semantic enrichment.

语义注释是现有文本接收标记的过程，该标记允许自动识别已命名的实体（例如，区分火鸡和土耳其）。手动注释既可以用作特定于域的设置中的独立过程，也可以用作自动注释算法的后处理。大多数注释工具的开发都着重于提供新颖的功能-最终用户对界面和用户交互的评估很少。

本文报告了一系列用户研究的结果，这些研究旨在探索非专家用户如何理解使用语义注释丰富文本的过程。我们发现这些用户可以轻松地创建简单的语义注释（例如，将概念分配给文本段落），但是很难理解复杂的语义注释（例如，将语义标识符分配给文本段落）。我们介绍了三项有关手动语义注释的用户研究的结果，并讨论了有关语义丰富过程以及我们将非专家暴露给语义丰富的方法的经验教训。