Open-domain question answering has exploded in popularity recently due to the success of dense retrieval models, which have surpassed sparse models using only a few supervised training examples. However, in this paper, we demonstrate current dense models are not yet the holy grail of retrieval. We first construct EntityQuestions, a set of simple, entity-rich questions based on facts from Wikidata (e.g., "Where was Arve Furset born?"), and observe that dense retrievers drastically underperform sparse methods. We investigate this issue and uncover that dense retrievers can only generalize to common entities unless the question pattern is explicitly observed during training. We discuss two simple solutions towards addressing this critical problem. First, we demonstrate that data augmentation is unable to fix the generalization problem. Second, we argue a more robust passage encoder helps facilitate better question adaptation using specialized question encoders. We hope our work can shed light on the challenges in creating a robust, universal dense retriever that works well across different input distributions.

中文翻译：

---

以实体为中心的简单问题挑战密集检索器

由于密集检索模型的成功，开放域问答最近大受欢迎，这些模型仅使用少数监督训练示例就超过了稀疏模型。然而，在本文中，我们证明当前的密集模型还不是检索的圣杯。我们首先构建了 EntityQuestions，这是一组基于 Wikidata 事实的简单、实体丰富的问题​​（例如，“Arve Furset 在哪里出生？”），并观察到密集检索器的性能明显低于稀疏方法。我们调查了这个问题并发现密集检索器只能泛化到常见的实体，除非在训练期间明确观察到问题模式。我们讨论了解决这个关键问题的两个简单的解决方案。首先，我们证明数据增强无法解决泛化问题。其次，我们认为更强大的段落编码器有助于使用专门的问题编码器更好地适应问题。我们希望我们的工作能够阐明在创建一个健壮的、通用的密集检索器方面的挑战，该检索器在不同的输入分布上都能很好地工作。