Inferences about hypotheses are ubiquitous in the cognitive sciences. Bayes factors provide one general way to compare different hypotheses by their compatibility with the observed data. Those quantifications can then also be used to choose between hypotheses. While Bayes factors provide an immediate approach to hypothesis testing, they are highly sensitive to details of the data/model assumptions and it’s unclear whether the details of the computational implementation (such as bridge sampling) are unbiased for complex analyses. Here, we study how Bayes factors misbehave under different conditions. This includes a study of errors in the estimation of Bayes factors; the first-ever use of simulation-based calibration to test the accuracy and bias of Bayes factor estimates using bridge sampling; a study of the stability of Bayes factors against different MCMC draws and sampling variation in the data; and a look at the variability of decisions based on Bayes factors using a utility function. We outline a Bayes factor workflow that researchers can use to study whether Bayes factors are robust for their individual analysis.

中文翻译：

---

用于稳健使用贝叶斯因子的工作流技术。

关于假设的推论在认知科学中无处不在。贝叶斯因子提供了一种通过与观察数据的兼容性来比较不同假设的通用方法。然后，这些量化也可用于在假设之间进行选择。虽然贝叶斯因子提供了一种直接的假设检验方法，但它们对数据/模型假设的细节高度敏感，并且不清楚计算实现的细节（例如桥接采样）对于复杂分析是否无偏见。在这里，我们研究了贝叶斯因子在不同条件下的行为异常。这包括研究贝叶斯因子估计中的误差；首次使用基于模拟的校准来测试使用桥采样的贝叶斯因子估计的准确性和偏差；研究贝叶斯因子对不同 MCMC 抽取的稳定性和数据中的抽样变化；并使用效用函数查看基于贝叶斯因子的决策的可变性。我们概述了一个贝叶斯因子工作流程，研究人员可以使用它来研究贝叶斯因子对于他们的个人分析是否稳健。