It is currently widely recognized that automated methods are crucial to help processing long-term recordings of marine bioacoustics. To evaluate the efficiency of such methods, it is essential to develop large-scale annotated datasets. However, besides being laborious and resource intensive, recent studies have suggested that such a task could also be highly subjective with the generation of annotator specific errors.

In this work, we investigate the question of inter-annotator agreement from a multi-annotator annotation campaign performed on a marine bioacoustics dataset. After providing quantitative evidence of inter-annotator variability, we investigate potential sources on both the user annotation practice and the annotation data and task to better understand why and how such variability occurs. Our study reveals that the acoustic event type, the Signal-to-Noise Ratio of the acoustic event and the annotator profile are three examples of critical factors impacting the annotation results of a multi-annotator campaign.

当前，广泛认可的是，自动化方法对于帮助处理海洋生物声学的长期记录至关重要。为了评估这种方法的效率，必须开发大规模的带注释的数据集。然而，除了费力和资源密集之外，最近的研究表明，随着注释器特定错误的产生，这样的任务也可能是高度主观的。

在这项工作中，我们通过对海洋生物声学数据集执行的多注释器注释活动来调查注释器之间的协议问题。在提供了批注者之间的可变性的定量证据之后，我们调查了用户批注实践以及批注数据和任务的潜在来源，以更好地理解这种可变性的发生原因和方式。我们的研究表明，声事件类型，声事件的信噪比和注释器配置文件是影响多注释器活动的注释结果的关键因素的三个示例。