Ethnicity-targeted hate speech has been widely shown to influence on-the-ground inter-ethnic conflict and violence, especially in such multi-ethnic societies as Russia. Therefore, ethnicity-targeted hate speech detection in user texts is becoming an important task. However, it faces a number of unresolved problems: difficulties of reliable mark-up, informal and indirect ways of expressing negativity in user texts (such as irony, false generalization and attribution of unfavored actions to targeted groups), users’ inclination to express opposite attitudes to different ethnic groups in the same text and, finally, lack of research on languages other than English. In this work we address several of these problems in the task of ethnicity-targeted hate speech detection in Russian-language social media texts. This approach allows us to differentiate between attitudes towards different ethnic groups mentioned in the same text – a task that has never been addressed before. We use a dataset of over 2,6M user messages mentioning ethnic groups to construct a representative sample of 12K instances (ethnic group, text) that are further thoroughly annotated via a special procedure. In contrast to many previous collections that usually comprise extreme cases of toxic speech, representativity of our sample secures a realistic and, therefore, much higher proportion of subtle negativity which additionally complicates its automatic detection. We then experiment with four types of machine learning models, from traditional classifiers such as SVM to deep learning approaches, notably the recently introduced BERT architecture, and interpret their predictions in terms of various linguistic phenomena. In addition to hate speech detection with a text-level two-class approach (hate, no hate), we also justify and implement a unique instance-based three-class approach (positive, neutral, negative attitude, the latter implying hate speech). Our best results are achieved by using fine-tuned and pre-trained RuBERT combined with linguistic features, with F1-hate=0.760, F1-macro=0.833 on the text-level two-class problem comparable to previous studies, and F1-hate=0.813, F1-macro=0.824 on our unique instance-based three-class hate speech detection task. Finally, we perform error analysis, and it reveals that further improvement could be achieved by accounting for complex and creative language issues more accurately, i.e., by detecting irony and unconventional forms of obscene lexicon.

针对种族的仇恨言论已被广泛证明会影响当地的种族间冲突和暴力，尤其是在俄罗斯这样的多种族社会中。因此，用户文本中针对种族的仇恨言论检测正成为一项重要的任务。然而，它面临着许多未解决的问题：可靠标记的困难、用户文本中表达消极情绪的非正式和间接方式（例如讽刺、错误概括和将不利行为归因于目标群体）、用户倾向于表达相反的观点同一文本中对不同民族的态度，最后，缺乏对英语以外的其他语言的研究。在这项工作中，我们解决了俄语社交媒体文本中针对种族的仇恨言论检测任务中的几个问题。这种方法使我们能够区分对同一文本中提到的不同种族群体的态度——这是以前从未解决过的任务。我们使用超过 2,600 万条提及族群的用户消息的数据集来构建 12,000 个实例（族群、文本）的代表性样本，这些实例通过特殊程序进一步彻底注释。与通常包含极端有害言论的许多以前的集合相比，我们样本的代表性确保了真实的，因此，微妙的消极性的比例要高得多，这也使其自动检测变得复杂。然后我们试验了四种类型的机器学习模型，从传统的分类器（如 SVM）到深度学习方法，特别是最近引入的 BERT 架构，并根据各种语言现象来解释他们的预测。除了使用文本级别的二类方法（仇恨，无仇恨）检测仇恨言论外，我们还证明并实施了一种独特的基于实例的三类方法（积极、中立、消极态度，后者暗示仇恨言论） . 我们的最佳结果是通过使用微调和预训练的 RuBERT 结合语言特征来实现的，F1-hate=0.760，F1-macro=0.833 在与之前的研究相当的文本级二分类问题上，F1-hate =0.813，F1-macro=0.824 在我们独特的基于实例的三类仇恨言论检测任务上。最后，我们进行了错误分析，它表明通过更准确地解释复杂和创造性的语言问题可以实现进一步的改进，即，