Testing for differences between two groups is among the most frequently carried out statistical methods in empirical research. The traditional frequentist approach is to make use of null hypothesis significance tests which use p values to reject a null hypothesis. Recently, a lot of research has emerged which proposes Bayesian versions of the most common parametric and nonparametric frequentist two-sample tests. These proposals include Student’s two-sample t-test and its nonparametric counterpart, the Mann–Whitney U test. In this paper, the underlying assumptions, models and their implications for practical research of recently proposed Bayesian two-sample tests are explored and contrasted with the frequentist solutions. An extensive simulation study is provided, the results of which demonstrate that the proposed Bayesian tests achieve better type I error control at slightly increased type II error rates. These results are important, because balancing the type I and II errors is a crucial goal in a variety of research, and shifting towards the Bayesian two-sample tests while simultaneously increasing the sample size yields smaller type I error rates. What is more, the results highlight that the differences in type II error rates between frequentist and Bayesian two-sample tests depend on the magnitude of the underlying effect.

检验两组之间的差异是经验研究中最常进行的统计方法之一。传统的常客主义方法是使用零假设显着性检验，该检验使用p否定原假设的值。最近，出现了许多研究，提出了最常见的参数化和非参数化的频繁样本两样本测试的贝叶斯版本。这些建议包括Student的两样本t检验和非参数对应的Mann-Whitney U检验。在本文中，对最近提出的贝叶斯两样本检验的基本假设，模型及其对实际研究的意义进行了探讨，并与频频解决方案进行了对比。提供了广泛的仿真研究，其结果表明，提出的贝叶斯测试在II类错误率略有提高的情况下实现了更好的I类错误控制。这些结果非常重要，因为平衡I型和II型错误是各种研究的关键目标，并转向贝叶斯两样本测试，同时增加样本数量会产生较小的I型错误率。更重要的是，结果强调，频繁样本和贝叶斯两次样本检验之间的II型错误率差异取决于潜在效应的大小。