Evidence retrieval is a key component of explainable question answering (QA). We argue that, despite recent progress, transformer network-based approaches such as universal sentence encoder (USE-QA) do not always outperform traditional information retrieval (IR) methods such as BM25 for evidence retrieval for QA. We introduce a lexical probing task that validates this observation: we demonstrate that neural IR methods have the capacity to capture lexical differences between questions and answers, but miss obvious lexical overlap signal. Learning from this probing analysis, we introduce a hybrid approach for evidence retrieval that combines the advantages of both IR directions. Our approach uses a routing classifier that learns when to direct incoming questions to BM25 vs. USE-QA for evidence retrieval using very simple statistics, which can be efficiently extracted from the top candidate evidence sentences produced by a BM25 model. We demonstrate that this hybrid evidence retrieval generally performs better than either individual retrieval strategy on three QA datasets: OpenBookQA, ReQA SQuAD, and ReQA NQ. Furthermore, we show that the proposed routing strategy is considerably faster than neural methods, with a runtime that is up to 5 times faster than USE-QA.

中文翻译：

---

仅在指甲上使用锤子：一种用于问答证据检索的混合方法

证据检索是可解释问答 (QA) 的关键组成部分。我们认为，尽管最近取得了进展，但基于 Transformer 网络的方法（例如通用句子编码器（USE-QA））并不总是优于传统的信息检索（IR）方法，例如用于 QA 证据检索的 BM25。我们引入了一个词汇探测任务来验证这一观察结果：我们证明神经 IR 方法有能力捕捉问题和答案之间的词汇差异，但会遗漏明显的词汇重叠信号。从这种探索性分析中学习，我们引入了一种结合了两个 IR 方向优势的证据检索混合方法。我们的方法使用路由分类器，该分类器使用非常简单的统计数据学习何时将传入的问题定向到 BM25 与 USE-QA 以进行证据检索，可以有效地从 BM25 模型生成的顶级候选证据句子中提取。我们证明，这种混合证据检索在三个 QA 数据集上的性能通常比单个检索策略更好：OpenBookQA、ReQA SQuAD 和 ReQA NQ。此外，我们表明所提出的路由策略比神经方法快得多，运行时间比 USE-QA 快 5 倍。