当前位置: X-MOL 学术arXiv.cs.IR › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Using the Hammer Only on Nails: A Hybrid Method for Evidence Retrieval for Question Answering
arXiv - CS - Information Retrieval Pub Date : 2020-09-22 , DOI: arxiv-2009.10791
Zhengzhong Liang, Yiyun Zhao, Mihai Surdeanu

Evidence retrieval is a key component of explainable question answering (QA). We argue that, despite recent progress, transformer network-based approaches such as universal sentence encoder (USE-QA) do not always outperform traditional information retrieval (IR) methods such as BM25 for evidence retrieval for QA. We introduce a lexical probing task that validates this observation: we demonstrate that neural IR methods have the capacity to capture lexical differences between questions and answers, but miss obvious lexical overlap signal. Learning from this probing analysis, we introduce a hybrid approach for evidence retrieval that combines the advantages of both IR directions. Our approach uses a routing classifier that learns when to direct incoming questions to BM25 vs. USE-QA for evidence retrieval using very simple statistics, which can be efficiently extracted from the top candidate evidence sentences produced by a BM25 model. We demonstrate that this hybrid evidence retrieval generally performs better than either individual retrieval strategy on three QA datasets: OpenBookQA, ReQA SQuAD, and ReQA NQ. Furthermore, we show that the proposed routing strategy is considerably faster than neural methods, with a runtime that is up to 5 times faster than USE-QA.

中文翻译:

仅在指甲上使用锤子:一种用于问答证据检索的混合方法

证据检索是可解释问答 (QA) 的关键组成部分。我们认为,尽管最近取得了进展,但基于 Transformer 网络的方法(例如通用句子编码器(USE-QA))并不总是优于传统的信息检索(IR)方法,例如用于 QA 证据检索的 BM25。我们引入了一个词汇探测任务来验证这一观察结果:我们证明神经 IR 方法有能力捕捉问题和答案之间的词汇差异,但会遗漏明显的词汇重叠信号。从这种探索性分析中学习,我们引入了一种结合了两个 IR 方向优势的证据检索混合方法。我们的方法使用路由分类器,该分类器使用非常简单的统计数据学习何时将传入的问题定向到 BM25 与 USE-QA 以进行证据检索,可以有效地从 BM25 模型生成的顶级候选证据句子中提取。我们证明,这种混合证据检索在三个 QA 数据集上的性能通常比单个检索策略更好:OpenBookQA、ReQA SQuAD 和 ReQA NQ。此外,我们表明所提出的路由策略比神经方法快得多,运行时间比 USE-QA 快 5 倍。
更新日期:2020-09-24
down
wechat
bug