Abstract Identifying species interactions and detecting when ecological communities are structured by them is an important problem in ecology and biogeography. Ecologists have developed specialized statistical hypothesis tests to detect patterns indicative of community‐wide processes in their field data. In this respect, null model approaches have proved particularly popular. The freedom allowed in choosing the null model and statistic to construct a hypothesis test leads to a proliferation of possible hypothesis tests from which ecologists can choose to detect these processes. Here, we point out some serious shortcomings of a popular approach to choosing the best hypothesis for the ecological problem at hand that involves benchmarking different hypothesis tests by assessing their performance on artificially constructed data sets. Terminological errors concerning the use of Type I and Type II errors that underlie these approaches are discussed. We argue that the key benchmarking methods proposed in the literature are not a sound guide for selecting null hypothesis tests, and further, that there is no simple way to benchmark null hypothesis tests. Surprisingly, the basic problems identified here do not appear to have been addressed previously, and these methods are still being used to develop and test new null models and summary statistics, from quantifying community structure (e.g., nestedness and modularity) to analyzing ecological networks.

中文翻译：

---

对生态零模型进行基准测试的困难：对当前方法的评估

摘要 识别物种相互作用并检测它们何时构建生态群落是生态学和生物地理学中的一个重要问题。生态学家开发了专门的统计假设检验，以检测表明其实地数据中社区范围进程的模式。在这方面，零模型方法已被证明特别受欢迎。允许选择零模型和统计量来构建假设检验的自由导致可能的假设检验激增，生态学家可以从中选择检测这些过程。在这里，我们指出了为手头的生态问题选择最佳假设的流行方法的一些严重缺陷，该方法涉及通过评估不同假设检验在人工构建的数据集上的性能来对它们进行基准测试。讨论了有关使用这些方法的基础的 I 类和 II 类错误的术语错误。我们认为，文献中提出的关键基准测试方法并不是选择零假设检验的可靠指南，此外，没有简单的方法可以对零假设检验进行基准测试。令人惊讶的是，这里确定的基本问题以前似乎没有得到解决，这些方法仍然被用于开发和测试新的空模型和汇总统计，从量化社区结构（例如，嵌套性和模块化）到分析生态网络。此外，没有简单的方法可以对零假设检验进行基准测试。令人惊讶的是，这里确定的基本问题以前似乎没有得到解决，这些方法仍然被用于开发和测试新的空模型和汇总统计，从量化社区结构（例如，嵌套性和模块化）到分析生态网络。此外，没有简单的方法可以对零假设检验进行基准测试。令人惊讶的是，这里确定的基本问题以前似乎没有得到解决，这些方法仍然被用于开发和测试新的空模型和汇总统计，从量化社区结构（例如，嵌套性和模块化）到分析生态网络。