Fact verification has attracted a lot of attention in the machine learning and natural language processing communities, as it is one of the key methods for detecting misinformation. Existing large-scale benchmarks for this task have focused mostly on textual sources, i.e. unstructured information, and thus ignored the wealth of information available in structured formats, such as tables. In this paper we introduce a novel dataset and benchmark, Fact Extraction and VERification Over Unstructured and Structured information (FEVEROUS), which consists of 87,026 verified claims. Each claim is annotated with evidence in the form of sentences and/or cells from tables in Wikipedia, as well as a label indicating whether this evidence supports, refutes, or does not provide enough information to reach a verdict. Furthermore, we detail our efforts to track and minimize the biases present in the dataset and could be exploited by models, e.g. being able to predict the label without using evidence. Finally, we develop a baseline for verifying claims against text and tables which predicts both the correct evidence and verdict for 18% of the claims.

中文翻译：

---

狂热：对非结构化和结构化信息的事实提取和验证

事实验证在机器学习和自然语言处理社区中引起了很多关注，因为它是检测错误信息的关键方法之一。这项任务的现有大规模基准主要集中在文本源上，即非结构化信息，因此忽略了结构化格式（如表格）中可用的大量信息。在本文中，我们介绍了一个新的数据集和基准，即对非结构化和结构化信息的事实提取和验证 (FEVEROUS)，它包含 87,026 个经过验证的声明。每个声明都以句子和/或维基百科表格中的单元格形式的证据进行注释，以及一个标签，表明该证据是支持、反驳还是没有提供足够的信息来达成判决。此外，我们详细说明了我们为跟踪和最小化数据集中存在的偏差所做的努力，这些偏差可以被模型利用，例如能够在不使用证据的情况下预测标签。最后，我们开发了一个基线，用于根据文本和表格验证声明，它预测了 18% 的声明的正确证据和判决。