It is an obvious fact that the power of a test statistic is dependent upon the significance (alpha) level at which the test is performed. It is perhaps a less obvious fact that the relative performance of two statistics in terms of power is also a function of the alpha level. Through numerous personal discussions, we have noted that even some competent statisticians have the mistaken intuition that relative power comparisons at traditional levels such as α=0.05 will be roughly similar to relative power comparisons at very low levels, such as the level α=510−8, which is commonly used in genome-wide association studies. In this brief note, we demonstrate that this notion is in fact quite wrong, especially with respect to comparing tests with differing degrees of freedom. In fact, at very low alpha levels the cost of additional degrees of freedom is often comparatively low. Thus we recommend that statisticians exercise caution when interpreting the results of power comparison studies which use alpha levels that will not be used in practice.

中文翻译：

---

关于在低显着性水平上比较检验统计量的功效的说明

一个明显的事实是，检验统计量的功效取决于执行检验的显着性 (alpha) 水平。两个统计量在功效方面的相对表现也是 alpha 水平的函数，这可能是一个不太明显的事实。通过大量的个人讨论，我们注意到，即使是一些有能力的统计学家也存在错误的直觉，即传统水平（例如 α=0.05）的相对功效比较将与非常低水平（例如 α=510− 水平）的相对功效比较大致相似。 8，常用于全基因组关联研究。在这个简短的说明中，我们证明这个概念实际上是非常错误的，尤其是在比较具有不同自由度的测试方面。实际上，在非常低的 alpha 水平下，额外自由度的成本通常相对较低。因此，我们建议统计学家在解释功效比较研究的结果时要谨慎，因为这些研究使用了不会在实践中使用的 alpha 水平。