The tremendous rate with which data is generated and analysis methods emerge makes it increasingly difficult to keep track of their domain of applicability, assumptions, limitations, and consequently, of the efficacy and precision with which they solve specific tasks. Therefore, there is an increasing need for benchmarks, and for the provision of infrastructure for continuous method evaluation. APAeval is an international community effort, organized by the RNA Society in 2021, to benchmark tools for the identification and quantification of the usage of alternative polyadenylation (APA) sites from short-read, bulk RNA-sequencing (RNA-seq) data. Here, we reviewed 17 tools and benchmarked eight on their ability to perform APA identification and quantification, using a comprehensive set of RNA-seq experiments comprising real, synthetic, and matched 3′-end sequencing data. To support continuous benchmarking, we have incorporated the results into the OpenEBench online platform, which allows for continuous extension of the set of methods, metrics, and challenges. We envisage that our analyses will assist researchers in selecting the appropriate tools for their studies, while the containers and reproducible workflows could easily be deployed and extended to evaluate new methods or data sets.

中文翻译：

---

从 RNA-seq 数据中识别和量化多腺苷酸化位点的方法的可扩展基准测试

数据生成和分析方法的出现速度之快，使得跟踪其适用范围、假设、局限性以及它们解决特定任务的有效性和精度变得越来越困难。因此，对基准以及为连续方法评估提供基础设施的需求不断增加。APAeval 是一项国际社区努力，由 RNA Society 于 2021 年组织，旨在对用于从短读长、批量 RNA 测序 (RNA-seq) 数据中识别和量化替代多腺苷酸化 (APA) 位点使用情况的工具进行基准测试。在这里，我们使用一套全面的 RNA-seq 实验（包括真实的、合成的和匹配的 3' 端测序数据）回顾了 17 种工具，并对其中 8 种工具执行 APA 识别和定量的能力进行了基准测试。为了支持持续的基准测试，我们已将结果合并到 OpenEBench 在线平台中，该平台允许不断扩展方法、指标和挑战集。我们设想我们的分析将帮助研究人员为他们的研究选择合适的工具，而容器和可重复的工作流程可以轻松部署和扩展以评估新方法或数据集。