Abstract We report on the results of an annotation experiment comparing naïve and expert coders in a sense disambiguation task consisting in the assignment of function labels to discourse markers (e.g. well, but, I mean) in spoken French and English using a taxonomy specifically designed for speech. Our qualitative-quantitative assessment of its reliability led us to suggest fundamental revisions of the structure of the taxonomy, striving to find a better balance between reliability and granularity. The resulting model articulates two independent levels of annotation (domains and functions) which, once combined, provide a robust tool for the analysis of discourse markers and relate them to more general functions of spoken language.

中文翻译：

---

话语注释中的可靠性与粒度：什么是权衡？

摘要：我们报告了一项注释实验的结果，该注释实验比较了天真的编码人员和专家编码人员在某种意义上的歧义消除工作，该任务包括使用专门设计的分类法将口语和英语口语标记中的功能标签分配给话语标记（例如：言语。我们对其可靠性进行了定性定量评估，因此我们建议对分类法的结构进行基本修订，力求在可靠性和粒度之间找到更好的平衡。生成的模型阐明了两个独立的注释级别（域和功能），一旦组合在一起，便为分析话语标记提供了一个强大的工具，并将它们与口头语言的更通用功能相关联。