Language model (LM) has become a common method of transfer learning in Natural Language Processing (NLP) tasks when working with small labeled datasets. An LM is pretrained using an easily available large unlabelled text corpus and is fine-tuned with the labelled data to apply to the target (i.e., downstream) task. As an LM is designed to capture the linguistic aspects of semantics, it can be biased to linguistic features. We argue that exposing an LM model during fine-tuning to instances that capture diverse semantic aspects (e.g., topical, linguistic, semantic relations) present in the dataset will improve its performance on the underlying task. We propose a Mixed Aspect Sampling (MAS) framework to sample instances that capture different semantic aspects of the dataset and use the ensemble classifier to improve the classification performance. Experimental results show that MAS performs better than random sampling as well as the state-of-the-art active learning models to abuse detection tasks where it is hard to collect the labelled data for building an accurate classifier.

中文翻译：

---

主动学习有效地将迁移学习微调到下游任务

在处理小型标记数据集时，语言模型 (LM) 已成为自然语言处理 (NLP) 任务中迁移学习的常用方法。LM 使用易于获得的大型未标记文本语料库进行预训练，并使用标记数据进行微调以应用于目标（即下游）任务。由于 LM 旨在捕获语义的语言方面，因此它可能会偏向于语言特征。我们认为，在微调期间将 LM 模型暴露给捕获数据集中存在的不同语义方面（例如，主题、语言、语义关系）的实例将提高其在基础任务上的性能。我们提出了一个混合方面采样（MAS）框架来对捕获数据集不同语义方面的实例进行采样，并使用集成分类器来提高分类性能。