On social media platforms, hateful and offensive language negatively impact the mental well-being of users and the participation of people from diverse backgrounds. Automatic methods to detect offensive language have largely relied on datasets with categorical labels. However, comments can vary in their degree of offensiveness. We create the first dataset of English language Reddit comments that has \textit{fine-grained, real-valued scores} between -1 (maximally supportive) and 1 (maximally offensive). The dataset was annotated using \emph{Best--Worst Scaling}, a form of comparative annotation that has been shown to alleviate known biases of using rating scales. We show that the method produces highly reliable offensiveness scores. Finally, we evaluate the ability of widely-used neural models to predict offensiveness scores on this new dataset.

中文翻译：

---

Ruddit：英语 Reddit 评论的攻击性规范

在社交媒体平台上，仇恨和攻击性语言会对用户的心理健康和来自不同背景的人的参与产生负面影响。检测攻击性语言的自动方法在很大程度上依赖于具有分类标签的数据集。但是，评论的冒犯程度可能会有所不同。我们创建了第一个英文 Reddit 评论数据集，其 \textit{fine-grained, real-valued score} 在 -1（最大支持）和 1（最大攻击）之间。数据集使用 \emph{Best--Worst Scaling} 进行注释，这是一种比较注释形式，已被证明可以减轻使用评分量表的已知偏差。我们表明该方法产生高度可靠的攻击性分数。最后，我们评估了广泛使用的神经模型在这个新数据集上预测攻击性分数的能力。