Multiple comparisons tests (MCTs) include the statistical tests used to compare groups (treatments) often following a significant effect reported in one of many types of linear models. Due to a variety of data and statistical considerations, several dozen MCTs have been developed over the decades, with tests ranging from very similar to each other to very different from each other. Many scientific disciplines use MCTs, including >40,000 reports of their use in ecological journals in the last 60 years. Despite the ubiquity and utility of MCTs, several issues remain in terms of their correct use and reporting. In this study, we evaluated 17 different MCTs. We first reviewed the published literature for recommendations on their correct use. Second, we created a simulation that evaluated the performance of nine common MCTs. The tests examined in the simulation were those that often overlapped in usage, meaning the selection of the test based on fit to the data is not unique and that the simulations could inform the selection of one or more tests when a researcher has choices. Based on the literature review and recommendations: planned comparisons are overwhelmingly recommended over unplanned comparisons, for planned non-parametric comparisons the Mann-Whitney-Wilcoxon U test is recommended, Scheffé’s S test is recommended for any linear combination of (unplanned) means, Tukey’s HSD and the Bonferroni or the Dunn-Sidak tests are recommended for pairwise comparisons of groups, and that many other tests exist for particular types of data. All code and data used to generate this paper are available at: https://github.com/stevemidway/MultipleComparisons.

中文翻译：

---

比较多重比较：选择最佳多重比较检验的实用指南

多重比较检验 (MCT) 包括用于比较组（治疗）的统计检验，通常遵循在许多类型的线性模型之一中报告的显着效果。由于各种数据和统计考虑，几十年来已经开发了几十个 MCT，测试范围从彼此非常相似到彼此非常不同。许多科学学科都使用 MCT，包括过去 60 年在生态学期刊上使用 MCT 的超过 40,000 份报告。尽管 MCT 无处不在且实用，但在其正确使用和报告方面仍存在一些问题。在这项研究中，我们评估了 17 种不同的 MCT。我们首先回顾了已发表的文献，以获取有关其正确使用的建议。其次，我们创建了一个模拟来评估九种常见 MCT 的性能。模拟中检查的测试在使用中经常重叠，这意味着基于数据拟合的测试选择不是唯一的，当研究人员有选择时，模拟可以告知选择一个或多个测试。基于文献回顾和建议：计划比较比计划外比较更受欢迎，对于计划的非参数比较，建议使用 Mann-Whitney-Wilcoxon U 检验，对于（计划外）均值的任何线性组合，建议使用 Scheffé's S 检验，Tukey's建议将 HSD 和 Bonferroni 或 Dunn-Sidak 检验用于组的成对比较，并且还有许多其他检验可用于特定类型的数据。用于生成本文的所有代码和数据均可从以下网址获得：https://github.com/stevemidway/MultipleComparisons。