The study of inter-annotator agreement in musical pattern annotations has gained increased attention over the past few years. While expert annotations are often taken as the reference for evaluating pattern discovery algorithms, relying on just one reference is not usually sufficient to capture the complex musical relations between patterns. In this paper, we address the potential of digital annotation tools to enable large-scale annotations of musical patterns, by comparing datasets gathered with two recently developed digital tools. We investigate the influence of the tools and different annotator backgrounds on the annotation process by performing inter-annotator agreement analysis and feature-based analysis on the annotated patterns. We discuss implications for further adaptation of annotation tools, and the potential for deriving reference data from such rich annotation datasets for the evaluation of automatic pattern discovery algorithms in the future.

音乐模式注释中注释者间一致性的研究在过去几年中受到越来越多的关注。虽然专家注释通常被用作评估模式发现算法的参考，但仅依靠一个参考通常不足以捕捉模式之间复杂的音乐关系。在本文中，我们通过将收集到的数据集与两个最近开发的数字工具进行比较，探讨了数字注释工具在对音乐模式进行大规模注释方面的潜力。我们通过对注释模式进行注释者间一致性分析和基于特征的分析来研究工具和不同注释者背景对注释过程的影响。我们讨论了进一步适应注释工具的影响，