We analyze the complexity of Gibbs samplers for inference in crossed random effect models used in modern analysis of variance. We demonstrate that for certain designs the plain vanilla Gibbs sampler is not scalable, in the sense that its complexity is worse than proportional to the number of parameters and data. We thus propose a simple modification leading to a collapsed Gibbs sampler that is provably scalable. Although our theory requires some balancedness assumptions on the data designs, we demonstrate in simulated and real datasets that the rates it predicts match remarkably the correct rates in cases where the assumptions are violated. We also show that the collapsed Gibbs sampler, extended to sample further unknown hyperparameters, outperforms significantly alternative state of the art algorithms.

中文翻译：

---

交叉随机效应模型的可扩展推理

我们分析了 Gibbs 采样器在现代方差分析中使用的交叉随机效应模型中进行推理的复杂性。我们证明，对于某些设计，普通的 Gibbs 采样器是不可扩展的，因为它的复杂性比与参数和数据的数量成正比更糟糕。因此，我们提出了一个简单的修改，导致折叠的 Gibbs 采样器可证明是可扩展的。尽管我们的理论需要对数据设计进行一些平衡性假设，但我们在模拟和真实数据集中证明，在违反假设的情况下，它预测的比率与正确的比率非常匹配。我们还展示了折叠的 Gibbs 采样器，扩展到对更多未知的超参数进行采样，其性能明显优于其他最先进的算法。