The statistical literature is known to be inconsistent in the use of the terms ‘permutation test’ and ‘randomisation test’. Several authors successfully argue that these terms should be used to refer to two distinct classes of tests and that there are major conceptual differences between these classes. The present paper explains an important difference in mathematical reasoning between these classes: a permutation test fundamentally requires that the set of permutations has a group structure, in the algebraic sense; the reasoning behind a randomisation test is not based on such a group structure, and it is possible to use an experimental design that does not correspond to a group. In particular, we can use a randomisation scheme where the number of possible treatment patterns is larger than in standard experimental designs. This leads to exact p values of improved resolution, providing increased power for very small significance levels, at the cost of decreased power for larger significance levels. We discuss applications in randomised trials and elsewhere. Further, we explain that Fisher's famous Lady Tasting Tea experiment, which is commonly referred to as the first permutation test, is in fact a randomisation test. This distinction is important to avoid confusion and invalid tests.

中文翻译：

---

重新审视女士品茶以及排列测试和随机化测试之间的差异

众所周知，统计文献在术语“置换检验”和“随机化检验”的使用上是不一致的。几位作者成功地争辩说，这些术语应该用于指代两个不同类别的测试，并且这些类别之间存在主要的概念差异。本文解释了这些类别之间数学推理的一个重要区别：置换测试从根本上要求置换集具有代数意义上的群结构；随机化测试背后的推理不是基于这样的组结构，并且可以使用与组不对应的实验设计。特别是，我们可以使用随机化方案，其中可能的治疗模式数量大于标准实验设计中的数量。这导致精确提高分辨率的p值，为非常小的显着性水平提供更高的功率，但以降低更大显着性水平的功率为代价。我们讨论在随机试验和其他地方的应用。此外，我们解释说，Fisher 著名的 Lady Tasting Tea 实验（通常称为第一个排列测试）实际上是一个随机化测试。这种区别对于避免混淆和无效测试很重要。