当前位置:
X-MOL 学术
›
arXiv.cs.DL
›
论文详情
Our official English website, www.x-mol.net, welcomes your
feedback! (Note: you will need to create a separate account there.)
Overview of the TREC 2019 Fair Ranking Track
arXiv - CS - Digital Libraries Pub Date : 2020-03-25 , DOI: arxiv-2003.11650 Asia J. Biega, Fernando Diaz, Michael D. Ekstrand, Sebastian Kohlmeier
arXiv - CS - Digital Libraries Pub Date : 2020-03-25 , DOI: arxiv-2003.11650 Asia J. Biega, Fernando Diaz, Michael D. Ekstrand, Sebastian Kohlmeier
The goal of the TREC Fair Ranking track was to develop a benchmark for
evaluating retrieval systems in terms of fairness to different content
providers in addition to classic notions of relevance. As part of the
benchmark, we defined standardized fairness metrics with evaluation protocols
and released a dataset for the fair ranking problem. The 2019 task focused on
reranking academic paper abstracts given a query. The objective was to fairly
represent relevant authors from several groups that were unknown at the system
submission time. Thus, the track emphasized the development of systems which
have robust performance across a variety of group definitions. Participants
were provided with querylog data (queries, documents, and relevance) from
Semantic Scholar. This paper presents an overview of the track, including the
task definition, descriptions of the data and the annotation process, as well
as a comparison of the performance of submitted systems.
中文翻译:
TREC 2019公平排名轨道概览
TREC Fair Ranking 赛道的目标是开发一个基准,用于评估检索系统对不同内容提供商的公平性以及相关性的经典概念。作为基准测试的一部分,我们使用评估协议定义了标准化的公平性指标,并发布了公平排名问题的数据集。2019 年的任务侧重于根据查询重新排列学术论文摘要。目的是公平地代表在系统提交时未知的几个组中的相关作者。因此,该赛道强调了在各种组定义中具有强大性能的系统的开发。Semantic Scholar 为参与者提供了查询日志数据(查询、文档和相关性)。本文概述了轨道,包括任务定义,
更新日期:2020-03-27
中文翻译:
TREC 2019公平排名轨道概览
TREC Fair Ranking 赛道的目标是开发一个基准,用于评估检索系统对不同内容提供商的公平性以及相关性的经典概念。作为基准测试的一部分,我们使用评估协议定义了标准化的公平性指标,并发布了公平排名问题的数据集。2019 年的任务侧重于根据查询重新排列学术论文摘要。目的是公平地代表在系统提交时未知的几个组中的相关作者。因此,该赛道强调了在各种组定义中具有强大性能的系统的开发。Semantic Scholar 为参与者提供了查询日志数据(查询、文档和相关性)。本文概述了轨道,包括任务定义,