Biomedical text mining is becoming increasingly important as the number of biomedical documents grow rapidly. Deep learning has boosted the development of biomedical text mining models. However, as deep learning models require a large amount of training data, a hierarchical attention based transfer learning model is proposed in this paper for the question answering task in biomedical field which lacks of sufficient training data. We adopt BERT (Bidirectional Encoder Representation Transformers), which has the ability to learn from large-scale unsupervised data, to enrich the semantic representation in our model. Especially, the scaled dot-product attention mechanism captures the question interaction clues for passage encoding. The domain adaptation technique of fine-tuning is used to reinforce the performance, which penalizes the deviations from the source model's parameters and remembers the knowledge of source domain. We evaluate the system performance on the open data set of BioASQ-Task B. The results show that our system achieves the state-of-the-art performance without any handcrafted features and outperforms the best solution for factoid questions in 2016 and 2017 BioASQ-Task B.

中文翻译：

---

基于深度尺度点积注意力的生物医学问答领域适应模型

随着生物医学文档数量的快速增长，生物医学文本挖掘变得越来越重要。深度学习推动了生物医学文本挖掘模型的发展。然而，由于深度学习模型需要大量的训练数据，本文针对生物医学领域缺乏足够训练数据的问答任务提出了一种基于层次注意力的迁移学习模型。我们采用能够从大规模无监督数据中学习的 BERT（双向编码器表示变换器）来丰富我们模型中的语义表示。特别是，缩放点积注意力机制捕获了用于段落编码的问题交互线索。微调的域适应技术用于增强性能，它惩罚与源模型参数的偏差并记住源域的知识。我们在 BioASQ-Task B 的开放数据集上评估系统性能。 结果表明，我们的系统在没有任何手工制作特征的情况下实现了最先进的性能，并且在 2016 年和 2017 年 BioASQ-任务 B。