P-values are useful statistical measures of evidence against a null hypothesis. In contrast to other statistical estimates, however, their sample-to-sample variability is usually not considered or estimated, and therefore not fully appreciated. Via a systematic study of log-scale p-value standard errors, bootstrap prediction bounds, and reproducibility probabilities for future replicate p-values, we show that p-values exhibit surprisingly large variability in typical data situations. In addition to providing context to discussions about the failure of statistical results to replicate, our findings shed light on the relative value of exact p-values vis-a-vis approximate p-values, and indicate that the use of *, **, and *** to denote levels 0.05, 0.01, and 0.001 of statistical significance in subject-matter journals is about the right level of precision for reporting p-values when judged by widely accepted rules for rounding statistical estimates.

中文翻译：

---

P 值精度和再现性

P 值是针对零假设的证据的有用统计量度。然而，与其他统计估计相比，它们的样本间变异性通常不被考虑或估计，因此没有得到充分理解。通过对对数尺度 p 值标准误差、bootstrap 预测界限和未来重复 p 值的再现性概率的系统研究，我们表明 p 值在典型数据情况下表现出惊人的大可变性。除了为有关统计结果无法复制的讨论提供背景外，我们的发现还阐明了精确 p 值与近似 p 值的相对值，并表明使用 *、**、 *** 表示水平 0.05、0.01 和 0。