The lack of a commonly used benchmark data set (collection) such as (Super-)GLUE (Wang et al., 2018, 2019) for the evaluation of non-English pre-trained language models is a severe shortcoming of current English-centric NLP-research. It concentrates a large part of the research on English, neglecting the uncertainty when transferring conclusions found for the English language to other languages. We evaluate the performance of the German and multilingual BERT-based models currently available via the huggingface transformers library on the four tasks of the GermEval17 workshop. We compare them to pre-BERT architectures (Wojatzki et al., 2017; Schmitt et al., 2018; Attia et al., 2018) as well as to an ELMo-based architecture (Biesialska et al., 2020) and a BERT-based approach (Guhr et al., 2020). The observed improvements are put in relation to those for similar tasks and similar models (pre-BERT vs. BERT-based) for the English language in order to draw tentative conclusions about whether the observed improvements are transferable to German or potentially other related languages.

中文翻译：

---

使用德语预训练语言模型重新评估GermEval17

缺乏用于评估非英语预训练语言模型的常用基准数据集（集合），例如（Super-）GLUE（Wang等人，2018,2019），这是当前以英语为中心的严重缺陷NLP研究。它集中了大部分的英语研究，而忽略了将英语得出的结论转换为其他语言时的不确定性。在GermEval17研讨会的四个任务上，我们评估了目前可通过拥抱变压器库使用的基于德语和多语言BERT的模型的性能。我们将它们与BERT之前的体系结构（Wojatzki等人，2017; Schmitt等人，2018; Attia等人，2018）以及基于ELMo的体系结构（Biesialska等人，2020）和BERT进行了比较基于方法（Guhr et al。，2020）。