Abstract

In a celebrated 1996 article, Schervish showed that, for testing interval null hypotheses, tests typically viewed as optimal can be logically incoherent. Specifically, one may fail to reject a specific interval null, but nevertheless—testing at the same level with the same data—reject a larger null, in which the original one is nested. This result has been used to argue against the widespread practice of viewing p-values as measures of evidence. In the current work we approach tests of interval nulls using simple Bayesian decision theory, and establish straightforward conditions that ensure coherence in Schervish’s sense. From these, we go on to establish novel frequentist criteria—different to Type I error rate—that, when controlled at fixed levels, give tests that are coherent in Schervish’s sense. The results suggest that exploring frequentist properties beyond the familiar Neyman–Pearson framework may ameliorate some of statistical testing’s well-known problems.

摘要

在 1996 年的一篇著名文章中，Schervish 表明，对于检验区间零假设，通常被视为最优的检验在逻辑上可能是不连贯的。具体来说，一个人可能无法拒绝一个特定的区间空值，但尽管如此——在相同的水平上用相同的数据进行测试——拒绝一个更大的空值，其中嵌套了原始的空值。这一结果已被用来反对观看p的普遍做法- 价值作为证据的衡量标准。在当前的工作中，我们使用简单的贝叶斯决策理论来处理区间空值测试，并建立直接条件以确保 Schervish 意义上的连贯性。从这些开始，我们继续建立新的频率论标准——不同于 I 类错误率——当控制在固定水平时，给出的测试在 Schervish 的意义上是连贯的。结果表明，在熟悉的 Neyman-Pearson 框架之外探索频率论属性可能会改善一些统计检验的众所周知的问题。