Inference from limited data requires a notion of measure on parameter space, which is most explicit in the Bayesian framework as a prior distribution. Jeffreys prior is the best-known uninformative choice, the invariant volume element from information geometry, but we demonstrate here that this leads to enormous bias in typical high-dimensional models. This is because models found in science typically have an effective dimensionality of accessible behaviours much smaller than the number of microscopic parameters. Any measure which treats all of these parameters equally is far from uniform when projected onto the sub-space of relevant parameters, due to variations in the local co-volume of irrelevant directions. We present results on a principled choice of measure which avoids this issue, and leads to unbiased posteriors, by focusing on relevant parameters. This optimal prior depends on the quantity of data to be gathered, and approaches Jeffreys prior in the asymptotic limit. But for typical models this limit cannot be justified without an impossibly large increase in the quantity of data, exponential in the number of microscopic parameters.

中文翻译：

---

远离渐近

从有限的数据进行推断需要参数空间的度量概念，这在贝叶斯框架中作为先验分布最为明确。Jeffreys 先验是最著名的无信息选择，即来自信息几何的不变体积元素，但我们在这里证明这会导致典型高维模型中的巨大偏差。这是因为在科学中发现的模型通常具有可访问行为的有效维度，远小于微观参数的数量。由于不相关方向的局部共体积的变化，当投影到相关参数的子空间时，任何同等对待所有这些参数的度量都远非统一。我们提出了关于避免这个问题的原则性测量选择的结果，并导致无偏见的后验，通过关注相关参数。该最优先验取决于要收集的数据量，并在渐近极限内接近 Jeffreys 先验。但是对于典型的模型，如果数据量没有不可能的大幅增加，那么这个限制就无法证明是合理的，微观参数的数量是指数级的。