Although open-domain question answering (QA) draws great attention in recent years, it requires large amounts of resources for building the full system and is often difficult to reproduce previous results due to complex configurations. In this paper, we introduce SF-QA: simple and fair evaluation framework for open-domain QA. SF-QA framework modularizes the pipeline open-domain QA system, which makes the task itself easily accessible and reproducible to research groups without enough computing resources. The proposed evaluation framework is publicly available and anyone can contribute to the code and evaluations.

中文翻译：

---

SF-QA：用于开放域问答的简单公正的评估库

尽管开放域问答（QA）近年来引起了极大的关注，但它需要大量资源来构建整个系统，并且由于配置复杂，通常难以重现以前的结果。在本文中，我们介绍了SF-QA：开放域质量检查的简单公平的评估框架。SF-QA框架对管道开放域QA系统进行了模块化，这使得任务本身很容易访问，并且对于没有足够计算资源的研究小组来说也可以重现。提议的评估框架是公开可用的，任何人都可以为准则和评估做出贡献。