Document screening is a fundamental task within Evidence-based Medicine (EBM), a practice that provides scientific evidence to support medical decisions. Several approaches have tried to reduce physicians’ workload of screening and labeling vast amounts of documents to answer clinical questions. Previous works tried to semi-automate document screening, reporting promising results, but their evaluation was conducted on small datasets, which hinders generalization. Moreover, recent works in natural language processing have introduced neural language models, but none have compared their performance in EBM. In this paper, we evaluate the impact of several document representations such as TF-IDF along with neural language models (BioBERT, BERT, Word2Vec, and GloVe) on an active learning-based setting for document screening in EBM. Our goal is to reduce the number of documents that physicians need to label to answer clinical questions. We evaluate these methods using both a small challenging dataset (CLEF eHealth 2017) as well as a larger one but easier to rank (Epistemonikos). Our results indicate that word as well as textual neural embeddings always outperform the traditional TF-IDF representation. When comparing among neural and textual embeddings, in the CLEF eHealth dataset the models BERT and BioBERT yielded the best results. On the larger dataset, Epistemonikos, Word2Vec and BERT were the most competitive, showing that BERT was the most consistent model across different corpuses. In terms of active learning, an uncertainty sampling strategy combined with a logistic regression achieved the best performance overall, above other methods under evaluation, and in fewer iterations. Finally, we compared the results of evaluating our best models, trained using active learning, with other authors methods from CLEF eHealth, showing better results in terms of work saved for physicians in the document-screening task.

中文翻译：

---

在主动学习环境中使用单词和文本嵌入自动筛选医学文献

文件筛选是循证医学 (EBM) 中的一项基本任务，EBM 是一种为支持医疗决策提供科学证据的实践。有几种方法试图减少医生筛选和标记大量文件以回答临床问题的工作量。以前的工作试图半自动化文档筛选，报告有希望的结果，但他们的评估是在小数据集上进行的，这阻碍了泛化。此外，最近在自然语言处理方面的工作引入了神经语言模型，但没有人比较它们在 EBM 中的性能。在本文中，我们评估了几种文档表示（例如 TF-IDF）以及神经语言模型（BioBERT、BERT、Word2Vec 和 GloVe）对 EBM 中基于主动学习的文档筛选设置的影响。我们的目标是减少医生需要标记以回答临床问题的文档数量。我们使用具有挑战性的小型数据集 (CLEF eHealth 2017) 以及更大但更易于排名的数据集 (Epistemonikos) 来评估这些方法。我们的结果表明，单词和文本神经嵌入总是优于传统的 TF-IDF 表示。在神经嵌入和文本嵌入之间进行比较时，在 CLEF eHealth 数据集中，模型 BERT 和 BioBERT 产生了最好的结果。在更大的数据集上，Epistemonikos、Word2Vec 和 BERT 是最具竞争力的，这表明 BERT 是跨不同语料库最一致的模型。在主动学习方面，与逻辑回归相结合的不确定性抽样策略总体上取得了最佳性能，高于其他评估方法，并且迭代次数更少。最后，我们将使用主动学习训练的最佳模型的评估结果与来自 CLEF eHealth 的其他作者方法进行了比较，在文档筛选任务中为医生节省的工作方面显示出更好的结果。