Experimental results are often interpreted through statistical tests, where the alternative hypothesis represents the theory to be evinced; if the experimental results lead to the rejection of the null hypothesis, the theory is supported by empirical evidence. In these cases, the reproducibility of this empirical evidence can be measured by the Reproducibility Probability (RP) of the test, which coincides with the probability of rejecting the null hypothesis. The terminology “Reproducibility” Probability stems from the fact that it is usually computed when an experiment provides a significant result to evaluate the probability that a further identical and independent experiment confirms the statistical significance. In recent literature, some RP estimators have been proposed. They are useful for two reasons: they allow us to evaluate the reliability of the obtained statistical significance and some estimates can be used as a test statistic, owing to the so-called “RP-testing” decision rule (reject the null hypothesis if and only if the RP estimate is greater than 1/2). Unfortunately, the usually adopted RP estimators are affected by a high mean squared error. In this paper, a new class of RP estimators is introduced and examined to improve their estimation precision. Specifically, the performances of the new RP estimators have been compared with those of the existing estimators and a 30% greater reduction in the mean squared error (on average) was observed. Moreover, the new estimator with the best performance allowed the use of the RP-testing decision rule. Hence, this work achieves the double goal of improving Reproducibility Probability estimation and preserving RP-testing.

实验结果通常通过统计检验来解释，其中备选假设代表了需要证明的理论；如果实验结果导致对原假设的否定，那么该理论将得到经验证据的支持。在这些情况下，可以通过测试的可重复性概率（RP）来衡量该经验证据的可重复性，这与否定原假设的概率相符。术语“可再现性”概率源于以下事实：通常是在实验提供重要结果以评估进一步相同且独立的实验确认统计显着性的概率时才进行计算。在最近的文献中，一些RP已经提出了估计量。它们之所以有用，有两个原因：它们使我们能够评估获得的统计显着性的可靠性，并且由于所谓的“ RP检验”决策规则，某些估计值可以用作检验统计量（如果和仅当RP估算值大于1/2时）。不幸的是，通常采用的RP估计量会受到较高的均方误差的影响。本文介绍了一种新型的RP估计量，并对其进行了研究以提高其估计精度。具体来说，新RP的性能估计值已与现有估计值进行了比较，并且观察到均方误差（平均）降低了30％。而且，具有最佳性能的新估计器允许使用RP测试决策规则。因此，这项工作达到了提高可重复性概率估计和保留RP测试的双重目标。