Cupping scores from experts are extensively used in the coffee industry for a variety of applications, from quality control to judging coffee competitions. In this paper, we examined inter‐rater reliability (IRR) of “clean cup” ratings by coffee experts (“cuppers”) in two studies. In both studies, IRR reliability was found to be low, denoting a lack of concept alignment among experts. Remarkably, however, within‐assessor reproducibility was high, suggesting that expert cuppers have their own individual understanding of “clean cup.”

中文翻译：

---

咖啡专家对“清洁杯”评分的评估者间可靠性

来自专家的拔罐得分广泛用于咖啡行业，从质量控制到评判咖啡比赛，应有尽有。在本文中，我们在两项研究中检查了咖啡专家（“杯”）对“干净杯”评级的评定者间可靠性（IRR）。在两项研究中，IRR的可靠性均很低，这表明专家之间缺乏概念一致性。但是，值得注意的是，评估员内部的可重复性很高，这表明专业的拔罐者对“清洁杯”有自己的理解。