Citation recommendation is intended to assist researchers in the process of searching for relevant papers to cite by recommending appropriate citations for a given input text. Existing test collections for this task are noisy and unreliable since they are built automatically from parsed PDF papers. In this paper, we present our ongoing effort at creating a publicly available, manually annotated test collection for citation recommendation. We also conduct a series of experiments to evaluate the effectiveness of content-based baseline models on the test collection, providing results for future work to improve upon. Our test collection and code to replicate experiments are available at https://github.com/boudinfl/acm-cr

中文翻译：

---

ACM-CR：用于引文推荐的手动注释测试集

引文推荐旨在通过为给定的输入文本推荐适当的引文来帮助研究人员搜索要引用的相关论文。此任务的现有测试集嘈杂且不可靠，因为它们是从解析的 PDF 论文中自动构建的。在本文中，我们展示了我们在为引文推荐创建公开可用的、手动注释的测试集方面所做的持续努力。我们还进行了一系列实验来评估基于内容的基线模型对测试集的有效性，为未来的工作提供改进的结果。我们的测试集和用于复制实验的代码可在 https://github.com/boudinfl/acm-cr 获得