Effective standardisation of methodologies to analyse the microbiome is essential to the entire microbiome community. Despite the microbiome field being established for over a decade, there are no accredited or certified reference materials available to the wider community. In this study, we describe the development of the first reference reagents produced by the National Institute for Biological Standards and Control (NIBSC) for microbiome analysis by next-generation sequencing. These can act as global working standards and will be evaluated as candidate World Health Organization International Reference Reagents. We developed the NIBSC DNA reference reagents Gut-Mix-RR and Gut-HiLo-RR and a four-measure framework for evaluation of bioinformatics tool and pipeline bias. Using these reagents and reporting system, we performed an independent evaluation of a variety of bioinformatics tools by analysing shotgun sequencing and 16S rRNA sequencing data generated from the Gut-Mix-RR and Gut-HiLo-RR. We demonstrate that key measures of microbiome health, such as diversity estimates, are largely inflated by the majority of bioinformatics tools. Across all tested tools, biases were present, with a clear trade-off occurring between sensitivity and the relative abundance of false positives in the final dataset. Using commercially available mock communities, we investigated how the composition of reference reagents may impact benchmarking studies. Reporting measures consistently changed when the same bioinformatics tools were used on different community compositions. This was influenced by both community complexity and taxonomy of species present. Both NIBSC reference reagents, which consisted of gut commensal species, proved to be the most challenging for the majority of bioinformatics tools tested. Going forward, we recommend the field uses site-specific reagents of a high complexity to ensure pipeline benchmarking is fit for purpose. If a consensus of acceptable levels of error can be agreed on, widespread adoption of these reference reagents will standardise downstream gut microbiome analyses. We propose to do this through a large open-invite collaborative study for multiple laboratories in 2020.

中文翻译：

---

制定微生物学领域的标准。

对微生物组进行分析的方法的有效标准化对于整个微生物组社区至关重要。尽管微生物组领域已经建立了十多年，但是没有更广泛的社区获得认可或认证的参考材料。在这项研究中，我们描述了由国家生物标准和控制研究所（NIBSC）生产的第一种参考试剂的开发，该试剂用于通过下一代测序进行微生物组分析。这些可以作为全球工作标准，并将被评估为世界卫生组织的国际参考试剂。我们开发了NIBSC DNA参考试剂Gut-Mix-RR和Gut-HiLo-RR，以及用于评估生物信息学工具和流水线偏倚的四项措施框架。使用这些试剂和报告系统，我们通过分析由Gut-Mix-RR和Gut-HiLo-RR产生的shot弹枪测序和16S rRNA测序数据，对各种生物信息学工具进行了独立评估。我们证明，大多数生物信息学工具大大夸大了微生物组健康的关键指标，例如多样性估计。在所有测试的工具中，都存在偏差，最终数据集中的敏感性和假阳性的相对丰度之间存在明显的权衡。我们使用市售的模拟社区，研究了参考试剂的组成如何影响基准研究。当在不同的社区组成中使用相同的生物信息学工具时，报告措施会不断发生变化。这受到社区复杂性和现有物种分类的影响。两种由肠道共生物种组成的NIBSC参考试剂被证明对大多数测试的生物信息学工具最具挑战性。展望未来，我们建议现场使用高复杂性的现场专用试剂，以确保管道基准测试适合目的。如果可以就可接受的误差水平达成共识，则这些参考试剂的广泛采用将使下游肠道微生物组分析标准化。我们建议在2020年通过针对多个实验室的大型开放式邀请研究来实现这一目标。如果可以就可接受的误差水平达成共识，则这些参考试剂的广泛采用将使下游肠道微生物组分析标准化。我们建议在2020年通过针对多个实验室的大型开放式邀请研究来实现这一目标。如果可以就可接受的误差水平达成共识，则这些参考试剂的广泛采用将使下游肠道微生物组分析标准化。我们建议在2020年通过针对多个实验室的大型开放式邀请研究来实现这一目标。